Awesome Shell Power: 2>&1 | tee …

2007-04-20

Collect the stdout and stderr of a command into a single file and see the output interactively:

some command 2>&1 | tee err

I use this with make so often that I have a script called mk:

make "$@" 2>&1 | tee err

(and don’t forget, we always use "$@", never $*)

The output file is named err but would probably be better named ofile (suggested by Nick B in comments). I use err in deference to 15 years of my personal Unix tradition.

The Order of Redirections

Now let’s take pipe out of the equation for a moment and consider redirections alone. On my Mac OS X laptop the command ls -d /eta /etc produces the following output:

ls: /eta: No such file or directory
/etc

The “No such file or directory” message appears on stderr and “/etc” appears on stdout.

You can prove that to yourself by closing one or the other file descriptor. Play around with ls -d /eta /etc >&-, and ls -d /eta /etc 2>&- and convince yourself that this is so.

Shell redirections are processed from left to right; we can get the stderr of a command to appear on its stdout and get rid of its stdout entirely:

$ ls -d /eta /etc 2>&1 1>&-
ls: /eta: No such file or directory

You might like to think about why this works. fd 2 is redirected to fd 1, meaning output to fd 2 goes to the same place as fd 1 at the time that the redirection was made. Then fd 1 is closed; this doesn’t affect the previous redirection of fd 2.

Doing it the other way round, we get this:

$ ls -d /eta /etc 1>&- 2>&1
-bash: 1: Bad file descriptor

That’s because by the time that it comes to redirect fd 2 to fd 1, fd 1 has been closed and can’t be used for a redirect.

How can we tell that 2>&1 1>&- sends what normally appears on stderr to stdout? Does this convince you:

$ ( ls -d /eta /etc 2>&1 1>&- ) | sed 's/^/*** /'
*** ls: /eta: No such file or directory

By borrowing an extra fd you can even swap stderr and stdout over:

$ ls -d /eta /etc 3>&2 2>&1 1>&3 3>&- | sed 's/^/*** /'
/etc
*** ls: /eta: No such file or directory

Notice that the error message from ls has gone through the pipe and been transformed by sed, so it must have been sent to stdout. The normal output of ls, “/etc”, has not gone through the pipe, so must’ve been sent to stderr.

About these ads

11 Responses to “Awesome Shell Power: 2>&1 | tee …”

  1. Richard Says:

    Nice!

    I find it’s clearer if I’m explicit about which of the two end-points of each ‘direction’ — the fd, or the stream — I am talking about. So instead of “fd 2 is redirected to fd 1″, I would say “fd 2 is redirected to the stream that fd 1 is currently directed to”. You are not actually redirecting one fd to another fd. (If I understand correctly).

  2. glorkspangle Says:

    The name “err” in “tee err” suggests that the output is errors only; better if it was “ofile” or something.

  3. glorkspangle Says:

    And hey, reading man sh(1) I discover that /bin/sh is tab-sensitive. I never knew. Another braindead tool.

  4. drj11 Says:

    Richard, yes that seems clearer. And you are correct.

    It’s all done using dup2(2). In the old days it was done using dup(2) and the knowledge that your new filedescriptor was always the lowest available positive integer.

    It’s not possible (as far as I’m aware) to chain one descriptor to another.

    There’s a subtlety when two descriptors refer to the same “stream” as you put it. The file pointer within that stream is shared.

  5. drj11 Says:

    How is /bin/sh tab-sensitive? That does/would suck.

  6. Gareth Rees Says:

    The only think I can think of is tab-removal in here-documents introduced by <<-

  7. drj11 Says:

    Never knew about <<- (and didn’t get that far down the page before getting bored). I feel slightly ill now.

  8. glorkspangle Says:

    Yes, it’s leading tab removal in

  9. glorkspangle Says:

    argh, <<- here documents.

  10. YAJer Says:

    Note also that you can make these redirects persistent in your shell using exec without a filename. This is sometimes handy for shell scripts.

  11. drj11 Says:

    Indeed. I’ve often used that useful exec feature.


Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out / Change )

Twitter picture

You are commenting using your Twitter account. Log Out / Change )

Facebook photo

You are commenting using your Facebook account. Log Out / Change )

Google+ photo

You are commenting using your Google+ account. Log Out / Change )

Connecting to %s

Follow

Get every new post delivered to your Inbox.

%d bloggers like this: