While Sun might have wanted us to believe “The Network is The Computer” the truth is that we often only need the network for access to stuff a brief points in time, and can get away with doing our real work on our local machines.  One of the systems at work went down today and is still currently avaiable.  This machine was exported several directories which I and some of my co-workers have NFS mounted.  When it failed, basica utilities on my machine were no longer functioning.  I tried several simple commands: like clear, ls, and which.

The problem was that my PATH environment variable had the remote machine before local machines.  This was required by our build process so that things like make would resolve to the correct version, consistent  across all machines.  When you type a command in at the command line, the shell resolves the command by trying each directory specified by the PATH variable.  In this case, the very first directory was not only failing, but hanging.

One trick that helped to confirm the problem was using echo * on a directory to see what files were there.  Since the command echo is built in to the shell, it does not cause any path navigation.  To view /usr/bin you execute echo /usr/bin/*.

To work around the problem export PATH=/bin:/usr/bin:$PATH. With that, basic utilities are once again resolved on the local machine.  Once the NFS server comes back up, exit out of you shell and create a new one, or re-export your path once again.