35.21 Using IFS to Split Strings

It might not be obvious why the Bourne shell has an IFS (internal field separator) shell variable. By default, it holds three characters: SPACE, TAB, and NEWLINE. These are the places that the shell parses command lines. So what?

If you have a line of text - say, from a database - and you want to split it into fields, the IFS variable can help. Put the field separator into IFS temporarily, use the shell's set (44.19) command to store the fields in command-line parameters; then restore the old IFS.

For example, the chunk of a shell script below gets current terminal settings from stty -g (42.4), which looks like this:

2506:5:bf:8a3b:3:1c:8:15:4:0:0:0:11:13:1a:19:12:f:17:16:0:0

The shell parses the line returned from stty by the backquotes (9.16). It stores x in $1. This trick stops errors if stty fails for some reason - without the x, if stty made no standard output, the shell's set command would print a list of all shell variables. Then 2506 goes into $2, 5 into $3, and so on. The original Bourne shell can only handle nine parameters (through $9); if your input lines may have more than nine fields, this isn't a good technique. But this script uses the Korn shell, which (along with bash) doesn't have that limit.

#!/bin/ksh
oldifs="$IFS"
# Change IFS to a colon:
IFS=:
# Put x in $1, stty -g output in $2 thru ${23}:
set x `stty -g`
IFS="$oldifs"
# Window size is in 16th field (not counting the first "x"):
echo "Your window has ${17} rows."

Because you don't need a subprocess to parse the output of stty, this can be faster than using an external command like cut (35.14) or awk (33.11).

There are places where IFS can't be used because the shell separates command lines at spaces before it splits at IFS. It doesn't split the results of variable substitution or command substitution (9.16) at spaces, though. Here's an example - three different ways to parse a line from /etc/passwd:

% cat splitter
#!/bin/sh
IFS=:
line='larry:Vk9skS323kd4q:985:100:Larry Smith:/u/larry:/bin/tcsh'
set x $line
echo "case 1: \$6 is '$6'"
set x `grep larry /etc/passwd`
echo "case 2: \$6 is '$6'"
set x larry:Vk9skS323kd4q:985:100:Larry Smith:/u/larry:/bin/tcsh
echo "case 3: \$6 is '$6'"

% ./splitter
case 1: $6 is 'Larry Smith'
case 2: $6 is 'Larry Smith'
case 3: $6 is 'Larry'

Case 1 used variable substitution and case 2 used command substitution; the sixth field contained the space. In case 3, though, with the colons on the command line, the sixth field was split: $6 became Larry and $7 was Smith. Another problem would have come up if any of the fields had been empty (as in larry::985:100:etc...)-the shell would "eat" the empty field and $6 would contain /u/larry. Using sed with its escaped parentheses (34.10) to do the searching and the parsing could solve the last two problems.