Advanced Bash-Scripting Guide: An in-depth exploration of the art of shell scripting
Prev		Next

Chapter 11. Internal Commands and Builtins

A builtin is a command contained within the Bash tool set, literally built in. This is either for performance reasons -- builtins execute faster than external commands, which usually require forking off a separate process -- or because a particular builtin needs direct access to the shell internals.

When a command or the shell itself initiates (or spawns) a new subprocess to carry out a task, this is called forking. This new process is the child, and the process that forked it off is the parent. While the child process is doing its work, the parent process is still executing.

Note that while a parent process gets the process ID of the child process, and can thus pass arguments to it, the reverse is not true. This can create problems that are subtle and hard to track down.

Example 11-1. A script that forks off multiple instances of itself

   1 #!/bin/bash
   2 # spawn.sh
   3 
   4 
   5 PIDS=$(pidof sh $0)  # Process IDs of the various instances of this script.
   6 P_array=( $PIDS )    # Put them in an array (why?).
   7 echo $PIDS           # Show process IDs of parent and child processes.
   8 let "instances = ${#P_array[*]} - 1"  # Count elements, less 1.
   9                                       # Why subtract 1?
  10 echo "$instances instance(s) of this script running."
  11 echo "[Hit Ctl-C to exit.]"; echo
  12 
  13 
  14 sleep 1              # Wait.
  15 sh $0                # Play it again, Sam.
  16 
  17 exit 0               # Not necessary; script will never get to here.
  18                      # Why not?
  19 
  20 #  After exiting with a Ctl-C,
  21 #+ do all the spawned instances of the script die?
  22 #  If so, why?
  23 
  24 # Note:
  25 # ----
  26 # Be careful not to run this script too long.
  27 # It will eventually eat up too many system resources.
  28 
  29 #  Is having a script spawn multiple instances of itself
  30 #+ an advisable scripting technique.
  31 #  Why or why not?

Generally, a Bash builtin does not fork a subprocess when it executes within a script. An external system command or filter in a script usually will fork a subprocess.

A builtin may be a synonym to a system command of the same name, but Bash reimplements it internally. For example, the Bash echo command is not the same as /bin/echo, although their behavior is almost identical.
1 #!/bin/bash 2 3 echo "This line uses the \"echo\" builtin." 4 /bin/echo "This line uses the /bin/echo system command."

A keyword is a reserved word, token or operator. Keywords have a special meaning to the shell, and indeed are the building blocks of the shell's syntax. As examples, "for", "while", "do", and "!" are keywords. Similar to a builtin, a keyword is hard-coded into Bash, but unlike a builtin, a keyword is not by itself a command, but part of a larger command structure. [1]

I/O

echo

prints (to stdout) an expression or variable (see Example 4-1).
1 echo Hello 2 echo $a

An echo requires the -e option to print escaped characters. See Example 5-2.

Normally, each echo command prints a terminal newline, but the -n option suppresses this.

An echo can be used to feed a sequence of commands down a pipe.

1 if echo "$VAR" | grep -q txt # if [[ $VAR = *txt* ]] 2 then 3 echo "$VAR contains the substring sequence \"txt\"" 4 fi

An echo, in combination with command substitution can set a variable.

a=`echo "HELLO" | tr A-Z a-z`

Be aware that echo `command` deletes any linefeeds that the output of command generates.

The $IFS (internal field separator) variable normally contains \n (linefeed) as one of its set of whitespace characters. Bash therefore splits the output of command at linefeeds into arguments to echo. Then echo outputs these arguments, separated by spaces.

bash$ ls -l /usr/share/apps/kjezz/sounds -rw-r--r-- 1 root root 1407 Nov 7 2000 reflect.au -rw-r--r-- 1 root root 362 Nov 7 2000 seconds.au bash$ echo `ls -l /usr/share/apps/kjezz/sounds` total 40 -rw-r--r-- 1 root root 716 Nov 7 2000 reflect.au -rw-r--r-- 1 root root 362 Nov 7 2000 seconds.au

So, how can we embed a linefeed within an echoed character string?
1 # Embedding a linefeed? 2 echo "Why doesn't this string \n split on two lines?" 3 # Doesn't split. 4 5 # Let's try something else. 6 7 echo 8 9 echo $"A line of text containing 10 a linefeed." 11 # Prints as two distinct lines (embedded linefeed). 12 # But, is the "$" variable prefix really necessary? 13 14 echo 15 16 echo "This string splits 17 on two lines." 18 # No, the "$" is not needed. 19 20 echo 21 echo "---------------" 22 echo 23 24 echo -n $"Another line of text containing 25 a linefeed." 26 # Prints as two distinct lines (embedded linefeed). 27 # Even the -n option fails to suppress the linefeed here. 28 29 echo 30 echo 31 echo "---------------" 32 echo 33 echo 34 35 # However, the following doesn't work as expected. 36 # Why not? Hint: Assignment to a variable. 37 string1=$"Yet another line of text containing 38 a linefeed (maybe)." 39 40 echo $string1 41 # Yet another line of text containing a linefeed (maybe). 42 # ^ 43 # Linefeed becomes a space. 44 45 # Thanks, Steve Parker, for pointing this out.

This command is a shell builtin, and not the same as /bin/echo, although its behavior is similar.

bash$ type -a echo echo is a shell builtin echo is /bin/echo

printf

The printf, formatted print, command is an enhanced echo. It is a limited variant of the C language printf() library function, and its syntax is somewhat different.

printf format-string... parameter...

This is the Bash builtin version of the /bin/printf or /usr/bin/printf command. See the printf manpage (of the system command) for in-depth coverage.

Older versions of Bash may not support printf.

Example 11-2. printf in action

   1 #!/bin/bash
   2 # printf demo
   3 
   4 PI=3.14159265358979
   5 DecimalConstant=31373
   6 Message1="Greetings,"
   7 Message2="Earthling."
   8 
   9 echo
  10 
  11 printf "Pi to 2 decimal places = %1.2f" $PI
  12 echo
  13 printf "Pi to 9 decimal places = %1.9f" $PI  # It even rounds off correctly.
  14 
  15 printf "\n"                                  # Prints a line feed,
  16                                              # Equivalent to 'echo' . . .
  17 
  18 printf "Constant = \t%d\n" $DecimalConstant  # Inserts tab (\t).
  19 
  20 printf "%s %s \n" $Message1 $Message2
  21 
  22 echo
  23 
  24 # ==========================================#
  25 # Simulation of C function, sprintf().
  26 # Loading a variable with a formatted string.
  27 
  28 echo 
  29 
  30 Pi12=$(printf "%1.12f" $PI)
  31 echo "Pi to 12 decimal places = $Pi12"
  32 
  33 Msg=`printf "%s %s \n" $Message1 $Message2`
  34 echo $Msg; echo $Msg
  35 
  36 #  As it happens, the 'sprintf' function can now be accessed
  37 #+ as a loadable module to Bash,
  38 #+ but this is not portable.
  39 
  40 exit 0

Formatting error messages is a useful application of printf

1 E_BADDIR=65 2 3 var=nonexistent_directory 4 5 error() 6 { 7 printf "$@" >&2 8 # Formats positional params passed, and sends them to stderr. 9 echo 10 exit $E_BADDIR 11 } 12 13 cd $var || error $"Can't cd to %s." "$var" 14 15 # Thanks, S.C.

read

"Reads" the value of a variable from stdin, that is, interactively fetches input from the keyboard. The -a option lets read get array variables (see Example 26-6).

Example 11-3. Variable assignment, using read

   1 #!/bin/bash
   2 # "Reading" variables.
   3 
   4 echo -n "Enter the value of variable 'var1': "
   5 # The -n option to echo suppresses newline.
   6 
   7 read var1
   8 # Note no '$' in front of var1, since it is being set.
   9 
  10 echo "var1 = $var1"
  11 
  12 
  13 echo
  14 
  15 # A single 'read' statement can set multiple variables.
  16 echo -n "Enter the values of variables 'var2' and 'var3' (separated by a space or tab): "
  17 read var2 var3
  18 echo "var2 = $var2      var3 = $var3"
  19 # If you input only one value, the other variable(s) will remain unset (null).
  20 
  21 exit 0

A read without an associated variable assigns its input to the dedicated variable $REPLY.

Example 11-4. What happens when read has no variable

   1 #!/bin/bash
   2 # read-novar.sh
   3 
   4 echo
   5 
   6 # -------------------------- #
   7 echo -n "Enter a value: "
   8 read var
   9 echo "\"var\" = "$var""
  10 # Everything as expected here.
  11 # -------------------------- #
  12 
  13 echo
  14 
  15 # ------------------------------------------------------------------- #
  16 echo -n "Enter another value: "
  17 read           #  No variable supplied for 'read', therefore...
  18                #+ Input to 'read' assigned to default variable, $REPLY.
  19 var="$REPLY"
  20 echo "\"var\" = "$var""
  21 # This is equivalent to the first code block.
  22 # ------------------------------------------------------------------- #
  23 
  24 echo
  25 
  26 exit 0

Normally, inputting a \ suppresses a newline during input to a read. The -r option causes an inputted \ to be interpreted literally.

Example 11-5. Multi-line input to read

   1 #!/bin/bash
   2 
   3 echo
   4 
   5 echo "Enter a string terminated by a \\, then press <ENTER>."
   6 echo "Then, enter a second string, and again press <ENTER>."
   7 read var1     # The "\" suppresses the newline, when reading $var1.
   8               #     first line \
   9               #     second line
  10 
  11 echo "var1 = $var1"
  12 #     var1 = first line second line
  13 
  14 #  For each line terminated by a "\"
  15 #+ you get a prompt on the next line to continue feeding characters into var1.
  16 
  17 echo; echo
  18 
  19 echo "Enter another string terminated by a \\ , then press <ENTER>."
  20 read -r var2  # The -r option causes the "\" to be read literally.
  21               #     first line \
  22 
  23 echo "var2 = $var2"
  24 #     var2 = first line \
  25 
  26 # Data entry terminates with the first <ENTER>.
  27 
  28 echo 
  29 
  30 exit 0

The read command has some interesting options that permit echoing a prompt and even reading keystrokes without hitting ENTER.

1 # Read a keypress without hitting ENTER. 2 3 read -s -n1 -p "Hit a key " keypress 4 echo; echo "Keypress was "\"$keypress\""." 5 6 # -s option means do not echo input. 7 # -n N option means accept only N characters of input. 8 # -p option means echo the following prompt before reading input. 9 10 # Using these options is tricky, since they need to be in the correct order.

The -n option to read also allows detection of the arrow keys and certain of the other unusual keys.

Example 11-6. Detecting the arrow keys

   1 #!/bin/bash
   2 # arrow-detect.sh: Detects the arrow keys, and a few more.
   3 # Thank you, Sandro Magi, for showing me how.
   4 
   5 # --------------------------------------------
   6 # Character codes generated by the keypresses.
   7 arrowup='\[A'
   8 arrowdown='\[B'
   9 arrowrt='\[C'
  10 arrowleft='\[D'
  11 insert='\[2'
  12 delete='\[3'
  13 # --------------------------------------------
  14 
  15 SUCCESS=0
  16 OTHER=65
  17 
  18 echo -n "Press a key...  "
  19 # May need to also press ENTER if a key not listed above pressed.
  20 read -n3 key                      # Read 3 characters.
  21 
  22 echo -n "$key" | grep "$arrowup"  #Check if character code detected.
  23 if [ "$?" -eq $SUCCESS ]
  24 then
  25   echo "Up-arrow key pressed."
  26   exit $SUCCESS
  27 fi
  28 
  29 echo -n "$key" | grep "$arrowdown"
  30 if [ "$?" -eq $SUCCESS ]
  31 then
  32   echo "Down-arrow key pressed."
  33   exit $SUCCESS
  34 fi
  35 
  36 echo -n "$key" | grep "$arrowrt"
  37 if [ "$?" -eq $SUCCESS ]
  38 then
  39   echo "Right-arrow key pressed."
  40   exit $SUCCESS
  41 fi
  42 
  43 echo -n "$key" | grep "$arrowleft"
  44 if [ "$?" -eq $SUCCESS ]
  45 then
  46   echo "Left-arrow key pressed."
  47   exit $SUCCESS
  48 fi
  49 
  50 echo -n "$key" | grep "$insert"
  51 if [ "$?" -eq $SUCCESS ]
  52 then
  53   echo "\"Insert\" key pressed."
  54   exit $SUCCESS
  55 fi
  56 
  57 echo -n "$key" | grep "$delete"
  58 if [ "$?" -eq $SUCCESS ]
  59 then
  60   echo "\"Delete\" key pressed."
  61   exit $SUCCESS
  62 fi
  63 
  64 
  65 echo " Some other key pressed."
  66 
  67 exit $OTHER
  68 
  69 #  Exercises:
  70 #  ---------
  71 #  1) Simplify this script by rewriting the multiple "if" tests
  72 #+    as a 'case' construct.
  73 #  2) Add detection of the "Home," "End," "PgUp," and "PgDn" keys.

The -n option to read will not detect the ENTER (newline) key.

The -t option to read permits timed input (see Example 9-4).

The read command may also "read" its variable value from a file redirected to stdin. If the file contains more than one line, only the first line is assigned to the variable. If read has more than one parameter, then each of these variables gets assigned a successive whitespace-delineated string. Caution!

Example 11-7. Using read with file redirection

   1 #!/bin/bash
   2 
   3 read var1 <data-file
   4 echo "var1 = $var1"
   5 # var1 set to the entire first line of the input file "data-file"
   6 
   7 read var2 var3 <data-file
   8 echo "var2 = $var2   var3 = $var3"
   9 # Note non-intuitive behavior of "read" here.
  10 # 1) Rewinds back to the beginning of input file.
  11 # 2) Each variable is now set to a corresponding string,
  12 #    separated by whitespace, rather than to an entire line of text.
  13 # 3) The final variable gets the remainder of the line.
  14 # 4) If there are more variables to be set than whitespace-terminated strings
  15 #    on the first line of the file, then the excess variables remain empty.
  16 
  17 echo "------------------------------------------------"
  18 
  19 # How to resolve the above problem with a loop:
  20 while read line
  21 do
  22   echo "$line"
  23 done <data-file
  24 # Thanks, Heiner Steven for pointing this out.
  25 
  26 echo "------------------------------------------------"
  27 
  28 # Use $IFS (Internal Field Separator variable) to split a line of input to
  29 # "read", if you do not want the default to be whitespace.
  30 
  31 echo "List of all users:"
  32 OIFS=$IFS; IFS=:       # /etc/passwd uses ":" for field separator.
  33 while read name passwd uid gid fullname ignore
  34 do
  35   echo "$name ($fullname)"
  36 done </etc/passwd   # I/O redirection.
  37 IFS=$OIFS              # Restore original $IFS.
  38 # This code snippet also by Heiner Steven.
  39 
  40 
  41 
  42 #  Setting the $IFS variable within the loop itself
  43 #+ eliminates the need for storing the original $IFS
  44 #+ in a temporary variable.
  45 #  Thanks, Dim Segebart, for pointing this out.
  46 echo "------------------------------------------------"
  47 echo "List of all users:"
  48 
  49 while IFS=: read name passwd uid gid fullname ignore
  50 do
  51   echo "$name ($fullname)"
  52 done </etc/passwd   # I/O redirection.
  53 
  54 echo
  55 echo "\$IFS still $IFS"
  56 
  57 exit 0

Piping output to a read, using echo to set variables will fail.

Yet, piping the output of cat seems to work.

1 cat file1 file2 | 2 while read line 3 do 4 echo $line 5 done

However, as Bj