Appriciate Your GNU Tools

It is often remarked that one never knows what one has until it is lost. While I think that most people do believe something along those lines, I suspect that such feelings are normally reserved for the big things in life: security, companionship, et cetra.

Personally, get this feeling every time I log onto a proprietary UNIX machine. Perhaps you recognize the situation: for every one of your clever little shell incantations, the machine responds with a brutally cold, "illegal option." I know that this command worked yesterday!" you say to yourself.

And then it dawns on you. You were on the Linux box yesterday.

These are the times that make us realize just how good we have it. Not only have the fine people at the FSF provided us with tools which are free--as in software--but also tools which are superior with an incredible amount of robustness and functionality.

Case in point: Today I was writing a script for our users to generate public keys for use with ssh (scp). The ssh implementation on Tru64 uses a different key format than OpenSSH. Fine, no big deal. That's not where the problem was.

The problem was with humble little grep. I wanted to capture the name of the key created from the ssh-keygen command. With the GNU version it's easy to pick out a just a matching pattern, rather than the entire line: just use the -o option:

$ grep -o 'pattern'
looking
for a pattern
> pattern
find the pattern in the line
> pattern
many little patterns. find all of the patterns
> pattern
> pattern

On the Tru64 version, there's no such option. In fact, the number of available options (17) falls far short of those made available by the GNU version (41). I'm not going to argue whether all of those options are necessary, but you'll be happy they are there when you need them most.

Luckily perl is installed on this machine, so I was still able to solve the problem:

$ perl -ne '/id_dsa_\d+_[^.]+/ && print $&, qq/\n/'

Of course, this solution presents its own difficulties. Namely, you might be writing a script that needs to be maintained by someone that's taken a few Korn Shell classes, yet knows absolutly zero perl. Besides, this is only an example. Sometimes the workaround doesn't come so easily. (Not to mention that perl is under an open source license as well.)

So the moral of the story? Be greatful for what you have: especially fantastic software, provided to you for free by volunteers.

RPMs by Hand: Ouch!

Watching Debian's Advanced Package Tool (apt) at work is an awesome thing to behold. Armed with apt-get and friends, installing a new package is only ever a few keystrokes away:

# apt-get install mypackage

The above command will have apt happily run off, fetch the package from the repository and install it on the system. All the dependancy checking is taken care of--for the most part--automatically. But what does this have to do with the Redhat Package Manager (RPM)? Well, nothing actually.

RPM does not intend to solve the same problem as apt. It's actually analogous to dpkg, Debian's package format and associated tools. Therefore, you can't really fault rpm for the fact that

# rpm -i mypackage.rpm

will most likely come screaming back at you that you are missing a bunch of dependancies.

No. The gripe here is that you really need to have something like apt. If you have a system decended from Debian, based on dpkg, you'll most certainly have apt as well; this is not necessarily the case with some RedHat relatives. Many of the rpm based distros do in fact have an acceptable replacement, while others--*hack* *cough* SuSE--do not.

No. YaST does not count. I am (unfairly?) limiting the field to command line applications. There is no way I'm firing up cruddy old yast just to install a simple package.

This combined by the fact that that you have packages which depend not only on other packages, but also individual files, leaves you in a sticky situation. Thus you are forced to do something ugly and inefficient like the following:

$ find /path/to/rpms -name '*.rpm' | while read package
  do
    rpm -qpl "${package}" | while read file; do
      echo -e "${file}\t${package}"
    done
  done > files-per-package
$ grep 'somestupidlib.so' files-per-package

to find out what you actually need to install. Once you get that list you can go right back to it with

# rpm -i some-other-package-with-1x10^7-deps.rpm

fun no?

P.S. If any SuSE fans out there have a better way of doing this, I'd love to hear it. The Administration Guide provided with SuSE Linux Pro 9.1—I don't have the more recent ones handy—suggests doing something remarkably similar to the above.

It's All in the Shell

Shell scripts are supposed to make our lives easier, right? Scribble down a bit of bash magic and all your busy work will vanish faster than you can type #!/bin/bash. Too good to be true?

Well, perhaps, but it will make life much easier in the long run. As your scripts become more advanced, you'll spend less time doing boring repetitive tasks, and more time...eh, writing scripts.

The best way to start out is to just play around at the command line for a while. This will get you on your way, but here is a pair of tips that will help you train your scripting muscle:

• Read a man page every day. The man pages are there to help you, but you can't take advantage of them unless you read them. You'll thank yourself later.

  Seriously. I'm not kidding. Do it for pleasure. Do it for power. Do it for the babes. Let's face it, ladies dig geeks, and this command that will lead you to everlasting glory:

  man $(ls -1 /usr/share/man/man1 | \
  sed -nr "$(($RANDOM % $(ls /usr/share/man/man1/ | wc -l) +1 ))p" | \
    sed "s/\.1\.gz//")

  What the heck is that, you ask? That will give you the low-down on a random command. Just think of it as "word of the day" for geeks.

• Read other quality scripts. There are plenty of places you can look for them, but some good starting points are the scripts provided by your distribution (You are running an open source operating system, aren't you?). Try looking through the system startup scripts under /etc/init.d/. You can learn a lot of good tricks just by trying to follow along with the code.

"Stupid" SSH Tricks

SSH is typically used to log into a remote machine and start up a shell session. This lets users run command line programs just as if they were sitting at a local terminal. That is a very useful ability to have in and of itself. However, some of the coolest things you can do with SSH don't involve starting an interactive session.

Here's a shortened version of the options ssh takes:

localhost$ ssh [misc. options] [user@]host [commands]

Normally, you'll see something simple along the lines of

localhost$ ssh remotehost

but, you can tack on an arbitrary command afterward to run it remotely. Let's say you need to get a list of all the users on a particular machine; there's no need to pull up a remote bash session. Just run this off:

localhost$ ssh -x remotehost 'cut -d: -f1 /etc/passwd'

    ...
    gandalf
    frodo
    ...
    localhost$

As indicated in the last line, this will leave you in your original session on the local machine. The -x option is used to turn of x forwarding; this will speed things up significantly.

Just using this method opens up a lot of options. If you have your authentication keys set up correctly, you can start running commands like this in shell scripts as well.

Also, you can poll many machines all at once. Let's say you want to check your group memberships on a bunch of machines. Use your shell's looping mechanism to do the trick.

localhost$ for box in host0{1,2,3,4}; do
  ssh -x ${box} groups
done

Nice.

Bonus stupid trick: if you want to run a remote command that requires user interaction—perhaps an ncurses application—you can have ssh request a pseudo terminal. For instance, you can open an editor on a remote file thus:

localhost$ ssh -xt vi somefile.txt

I'm sure you can come up with more interesting examples.