Bash (Bourne-Again SHell) is a very powerful shell available on many operating systems. Combined with Unix-style commands, it is hard to find a repetitive task that can’t have its execution simplified. Recently, I was cleaning up some code and discovered duplication in two different files. Like a good engineer, I saw this as an opportunity to refactor and consolidate two files into a single file in a new location. As part of the refactoring, I needed to remove the old files since they are no longer needed. These files had the same name but were in different directories. The file structure of the project may look like the following:
/path/foo/baz
/path/bar/baz
The easiest approach would be to issue the rm command, either twice, or by supplying multiple file arguments. These approaches look like this:
rm /path/foo/baz
rm /path/bar/baz
or alternately
rm /path/foo/baz /path/bar/baz
This works fine for a handful of file names, but once we reach a certain amount of file names in each location (say 3), then this becomes unwieldy. If you find yourself doing repetitive tasks in Bash, you are probably not taking advantage of many of the constructs the shell, and Unix utilities it provides. Let’s look at an example with multiple repeat files in two different directories:
/path/foo/a
/path/foo/b
/path/foo/c
/path/bar/a
/path/bar/b
/path/bar/c
Our simple rm command becomes:
rm /path/foo/a /path/foo/b /path/foo/c /path/bar/a /path/bar/b /path/bar/c
Yikes! Bash to the rescue. Like any problem, there are several approaches we can take to cut down on the repetitive nature of this task.
History Expansion with Modifiers
rm /path/foo/a /path/foo/b /path/foo/c
!!:gs/foo/bar/
We first issue the rm command for the first set of files that we want to delete. After executing the first command, the event designator !! will refer to the previous command (rm). Following this command is a modifier that will allow us to do a substitution (globally). This will reissue the rm command, replacing the path foo with the path bar. An alternative syntax to this command is using the caret quick substitution event designator. The functionality is similar, however the replacement only matches the first occurrence. Note that referring to the previous command is implicit:
rm /path/foo/a /path/foo/b /path/foo/c
^foo^bar^
Brace Expansion
Bash allows for different types of expansions, one of which is the brace expansion. This can generate arbitrary strings using a combination of the options contained. We can remove from two directories at once by issuing the command:
rm /path/{foo,bar}/baz
Further, the number of brace expansions in a single command can increase the number of combinations. For example, we can delete multiple file names in multiple directories using the following command:
rm /path/{foo,bar}/{a,b,c}
Looping
Bash also supports scripting, and many programming structures can be invoked in the shell. We have the ability to use the for command and iterate through a collection, executing a command for each item in the collection. This is useful if we want to specify a part of the file path for deleting your records:
For f in "foo" "bar"; do rm /path/$f/a; done
Traditionally, we would see this broken into multiple lines; however, we can append additional commands using the semicolon. In this method, we build our path, and for each path part, we call rm.
Sed + Xargs
While there are more simple ways to tackle this problem, we can use a mix of Unix utilities inside our command. Sed is a stream editor that we can leverage for doing an in-place find and replace. Xargs allows us to build and execute commands from standard input. Let’s see an example of these tools working in tandem:
rm /path/foo/a
echo !$ | sed 's/foo/bar/' | xargs rm
First, we issue our delete request. On the subsequent command, we echo $!, which will be replaced in Bash by the argument to our previous command. We then pipe this output into sed, which allows us to do a find and replace on those arguments. We take this modified argument and send it to xargs to allow us to issue the rm command with the new path as its argument. This is certainly not the easiest way to accomplish this particular task, but it shows the power that Bash and Unix utilities provide to complete repetitive tasks. I highly recommend everyone glance at the Bash manual page, which can be displayed by typing man bash at a terminal.