9.21 Handle Too-Long Command Lines with xargs

xargs is one of those UNIX utilities that seems pretty useless when you first hear about it - but turns into one of the handiest tools you can have.

xargs	If your system doesn't already have xargs , be sure to install it from the CD-ROM.

xargs reads a group of arguments from its standard input, then runs a UNIX command with that group of arguments. It keeps reading arguments and running the command until it runs out of arguments. The shell's backquotes ( 9.16 ) do the same kind of thing, but they give all the arguments to the command at once. This can give you a Too many arguments ( 9.20 ) error.

Here are a couple of examples:

• If you want to print most of the files in a large directory, put the output of ls into a file. Edit the file to leave just the filenames you want printed. Give the file to xargs ' standard input:

  <	% ls > allfiles.tmp % vi allfiles.tmp % xargs lpr < allfiles.tmp

  What did that do? With lines like these in allfiles.tmp :

  % 
  
  cat allfiles.tmp
  
   afile application    ... yoyotest zapme

  xargs ran one or more lpr commands, each with a group of arguments, until it had read every word in the file:

  lpr afile application ...    ... lpr ... yoyotest zapme

• The standard output of xargs is the standard output of the commands it runs. So, if you'd created allfiles.tmp above but you wanted to format the files with pr ( 43.7 ) first, you could type:

  % 
  
  xargs pr < allfiles.tmp | lpr
  
  

  Then xargs would run all of these pr commands. The shell would pipe their standard outputs [4] to a single lpr command:

  [4] Actually, the shell is piping the standard output of xargs . As I said above, xargs sends the standard output of commands it runs to its own standard output.

  pr afile application ...    ... pr ... yoyotest zapme

  In this next example, find ( 17.1 ) gets a list of all files in the directory tree. Next, we use xargs to read those filenames and run grep -l ( 15.7 ) to find which files contain the word "WARNING." Next, we pipe that to a setup with pr and lpr , like the one in the previous example:

  % 
  
  find . -type f -print | xargs grep -l WARNING | xargs pr | lpr
  
  

  "Huh?" you might say. Just take that step by step. The output of find is a list of filenames, like ./afile ./bfile ... ./adir/zfile and so on. The first xargs gives those filenames to one or more grep -l commands:

  
  grep -l WARNING ./afile ./bfile ...    ... grep -l WARNING ./adir/zfile ...

  The standard output of all those grep s is a (shortened) list of filenames that match. That's piped to another xargs -it runs pr commands with the filenames that grep found.

  UNIX is weird and wonderful!

• Sometimes you don't want xargs to run its command with as many arguments as it can fit on the command line. The -n option sets the maximum number of arguments xargs will give to each command. Another handy option, -p , prompts you before running each command.

  Here's a directory full of files with errors (whose names end with .bad ) and corrected versions (named .fixed ). I use ls to give the list of files to xargs ; it reads two filenames at once, then asks whether I want to run diff -c to compare those two files. It keeps prompting me and running diff -c until it runs out of file pairs:

  % 
  
  ls
  
   chap1.bad chap1.fixed chap2.bad chap2.fixed     ... chap9.bad chap9.fixed % 
  
  ls | xargs -p -n2 diff -c
  
   diff -c chap1.bad chap1.fixed ?...
  
  y
  
      ...
  Output of diff command for chap1
  ... diff -c chap2.bad chap2.fixed ?...
  
  n
  
   diff -c chap3.bad chap3.fixed ?...
  
  y
  
      ...
  Output of diff command for chap3
  ...     ...

As the next article ( 9.22 ) explains, xargs can have trouble if an argument has white space inside a word. Luckily, the GNU xargs (read about it there) solves the problem.