18.16 Copying Directory Trees with (tar | tar)

The tar (19.5) command isn't just for tape archives. It can copy files from disk to disk, too. And even if your computer has cp -r (18.15), there are advantages to using tar.

The obvious way to copy directories with tar is to write them onto a tape archive with relative pathnames - then read back the tape and write it somewhere else on the disk. But tar can also write to a UNIX pipe - and read from a pipe. This looks like:

% reading-tar | writing-tar

with one trick: the writing-tar process has a different current directory (38.3, 38.4) (the place where you want the copy made) than the reading-tar. To do that, run the writing-tar in a subshell (13.7).

The argument(s) to the reading-tar can be directory(s) or file(s). Just be sure to use relative pathnames (14.2) that don't start with a slash - otherwise, the writing-tar will write the copies in the same place the originals came from!

"How about an example," you ask? Figure 18.2 has one. It copies from the directory /home/jane, with all its files and subdirectories. The copy is made in the directory /work/bkup/jane:

% mkdir /work/bkup/jane
% cd /home/jane
% tar cf - . | (cd /work/bkup/jane && tar xBf -)

The && operator (44.9) tells the shell to start tar xBf only if the previous command (the cd) succeeded. That prevents tar writing files into the same directory it's reading from - if the destination directory isn't accessible or you flub its pathname. If your tar has a B (reblocking) option, use it to help be sure that the copy is made correctly. If your tar doesn't have a reblocking option, you can use this trick suggested by Chris Torek:

% tar cf - . | cat | (cd /work/backup/jane && tar xbf 1 -)

CAUTION: At least one tar version has a v (verbose) option that writes the verbose text to standard output instead of standard error ! (19.8) If your tar does that, don't use v on the reading-tar (the tar that feeds the pipe)-use v on the writing-tar.

You can use other options that your tar might have-like excluding files or directories (20.8)- on the reading-tar, too. Some gotchas:

• Symbolic links (18.4) will be copied exactly. If they point to relative pathnames, the copied links might point to locations that don't exist. You can search for these symbolic links with find -type l (17.13) or oldlinks (16.28).

• A hard link (18.4) will be copied as a file. If there are more hard links to that file in the files you're copying, they will be linked to the copy of the first link. That can be good because the destination might be on a different filesystem (a hard link to the original file can't work then). It can be bad if the link pointed to a really big file; the copy can take a lot of disk space. You can search for these hard links by:

  • Searching the directory from which you're copying with find -links +1 -type f (17.5) to find all files that have more than one link, and

  • Running the reading-tar with its l (lowercase letter L) option to complain if it didn't copy all links to a file.

Figure 18.2: Copying /home/jane to /work/bkup with tar

If your system has rsh (1.33), you can run the writing-tar on a remote system. For example, to copy a directory to the computer named kumquat:

% rsh kumquat mkdir /work/bkup/jane
% tar cf - . | rsh kumquat 'cd /work/bkup/jane && tar xBf -'