16.14 Showing Non-Printable Characters in Filenames

From time to time, you may get files with non-printing characters, spaces, and other garbage in them. This usually is the result of some mistake - but it's a pain nevertheless.

If you're using BSD UNIX, the ls command gives you some help; it will convert all non-printing characters to a question mark (?), giving you some idea that something funny is there. [2] For example:

[2] The -q option is the default only when ls's standard output is a terminal. If you pipe the output or redirect it to a file, remember to add -q.

% ls
ab??cd

This shows that there are two non-printing characters between ab and cd. To delete (or rename) this file, you can use a wildcard pattern like ab??cd.

NOTE: BE CAREFUL. When I was new to UNIX, I once accidentally generated a lot of weird filenames. ls told me that they all began with ????, so I naively typed rm ????*. That's when my troubles began. See article 23.2 for the rest of the gruesome story. (I spent the next day and night trying to undo the damage.) THE MORAL IS: It's always a good idea to use echo to test filenames with wildcards in them.

If you're using System V UNIX, you have a different set of problems. System V's ls doesn't convert the non-printing characters to question marks. In fact, it doesn't do anything at all - it just spits these weird characters at your terminal, which can respond in any number of strange and hostile ways. Most of the non-printing characters have special meanings - ranging from "don't take any more input" to "clear the screen."

To prevent this, use the -b option. [3] This tells ls to print the octal value of any non-printing characters, preceeded by a backslash. For example, on System V:

[3] On BSD, pipe the ls -q output through cat -v or od -c (25.7) to see what the non-printing characters are.

% ls -b
ab\013\014cd

This shows that the non-printing characters have octal values 13 and 14, respectively. If you look up these values in an ASCII table (51.3), you will see that they correspond to CTRL-k and CTRL-l. And - if you think about what's happening - you'll realize that CTRL-l is a formfeed character, which tells many terminals to clear the screen. That's why the regular ls command behaved so strangely.

Once you know what you're dealing with, you can use a wildcard pattern to delete or rename the file.