35.20 Quick Reference: uniq

uniq is used to remove duplicate adjacent lines from a sorted file, sending one copy of each line to standard output or to a second file, if one is specified on the command line.

NOTE: Be warned:

% uniq file1 file2

will not print the unique lines from both file1 and file2 to standard output! It will replace the contents of file2 with the unique lines from file1.

uniq is often used as a filter (1.30). See also comm (28.12), sort (36.1), and especially sort -u (36.6).

options

-c       Print each line once, counting instances of each.
-d       Print duplicate lines once, but no unique lines.
-u       Print only unique lines (no copy of duplicate entries is kept).
-n       Ignore first n fields of a line. Fields are separated by spaces
or by tabs.
+n       Ignore first n characters of a field.

You can specify only one of the options -c, -d, or -u.

Examples

To send only one copy of each line from list to output file list.new:

uniq list list.new

To show which names appear more than once:

sort names | uniq -d

To show which lines appear exactly three times:

grep	sort names | uniq -c | grep " 3 "