Giving New Life to Bourne-Again SHell (BASH).

Bash (Bourne-Again SHell) is a very powerful shell available on many operating systems. Combined with Unix style commands, it is hard to find a repetitive task that can’t have its execution simplified. Recently I was cleaning up some code and discovered duplication in two different files. Like a good programmer, I saw this as an opportunity to refactor and consolidated the two files into a single file in a new location. As part of the refactoring, I needed to remove the old files, since they are no longer needed. These files had the same name but were in different directories. The file structure of the project may look like the following:

    /path/foo/baz
    /path/bar/baz

The easiest approach would be to issue the rm command, either twice, or by supplying multiple file arguments. These approaches look like this:

    rm /path/foo/baz
    rm /path/bar/baz

or alternately

    rm /path/foo/baz /path/bar/baz

This works fine for a handful of file names, but once we reach a certain amount of file names in each location (say 3), then this becomes unwieldy. If you find yourself doing repetitive tasks in Bash, you are probably not taking advantage of many of the constructs the shell, and Unix utilities provide. Let’s look at an example with multiple repeat files in two different directories:

    /path/foo/a
    /path/foo/b
    /path/foo/c

    /path/bar/a
    /path/bar/b
    /path/bar/c

Our simple rm command becomes:

rm /path/foo/a /path/foo/b /path/foo/c /path/bar/a /path/bar/b /path/bar/c

Yikes! Bash to the rescue. Like any problem, there are several approaches we can take to cut down on the repetitive nature of this task.

History Expansion with Modifiers

    rm /path/foo/a /path/foo/b /path/foo/c
    !!:gs/foo/bar/

We first issue the rm command for the first set of files that we want to delete. After executing the first command, the event designator !! will refer to the previous command (rm). Following this command is a modifier that will allow us to do a substitution (globally). This will reissue the rm command replacing the path foo with the path bar. An alternative syntax to this command is using the caret quick substitution event designator. The functionality is similar, however the replacement only matches the first occurrence. Note that referring to the previous command is implicit:

    rm /path/foo/a /path/foo/b /path/foo/c
    ^foo^bar^

Brace Expansion

Bash allows for different types of expansions, one of which being the brace expansion. This can generate arbitrary strings using a combination of the options contained. We can remove from two directories at once by issuing the command:

    rm /path/{foo,bar}/baz

Further, the number of brace expansions in a single command can increase the number of combinations. For example, we can delete multiple file names in multiple directories using the following command:

    rm /path/{foo,bar}/{a,b,c}

Looping

Bash also supports scripting, and many programming structures can be invoked in the shell. We have the ability to use the for command, and iterate through a collection, executing a command for each item in the collection. This is useful if we want to specify a part of the file path for deleting your records:

    for f in "foo" "bar"; do rm /path/$f/a; done

Traditionally, we would see this broken into multiple lines; however, we can append additional commands using the semicolon. In this method, we build our path, and for each path part, we call rm.

Sed + Xargs

While there are more simple ways to tackle this problem, we can use a mix of Unix utilities inside our command. Sed is a stream editor that we can leverage for doing an in place find and replace. Xargs allows us to build and execute commands from standard input. Let’s see an example of these tools working in tandem:

    rm /path/foo/a
    echo !$ | sed 's/foo/bar/' | xargs rm

First we issue our delete request. On the subsequent command, we echo $!, which will be replaced in Bash by the argument to our previous command. We then pipe this output into sed, which allows us to do a find and replace on those arguments. We take this modified argument and send it to xargs to allow us to issue the rm command with the new path as its argument. This is certainly not the easiest way to accomplish this particular task, but it shows the power that Bash and Unix utilities provide to complete repetitive tasks. I highly recommend everyone glance at the Bash manual page, which can be displayed by typing man bash at a terminal. An online copy can be found at http://linux.die.net/man/1/bash.

—Ben Simpson