Why stderr bypasses pipes—and how to capture it
When output passes through grep or a file but warnings still appear on screen, the shell is doing exactly what its descriptor wiring requests. Trace descriptors 1 and 2 to control both paths.
Reviewed by the Terminaster editorial team · Updated
What you will take away
- A normal pipe connects the left command’s stdout, descriptor 1, to the right command’s standard input.
- stderr is descriptor 2 and keeps its previous destination unless you redirect or merge it explicitly.
- Use separate files when automation needs parseable output and retained diagnostics.
- Read redirections left to right because 2>&1 copies stdout’s destination at that moment.
Choose whether grep or tee should see diagnostics
A normal A | B pipeline sends A’s stdout, descriptor 1, to B’s stdin while stderr, descriptor 2, keeps its existing destination. That is why grep can miss an error you still see on the terminal and why tee can save results without saving warnings.
Place 2>&1 on the left command when the consumer should receive both streams. Keep them separate when stdout is structured data: merging an error line into JSON, filenames, or a count can corrupt the downstream result.
Merge diagnostics only when the consumer should see them
command 2>&1 | grep 'pattern'
command 2>&1 | tee combined.log
# Bash shorthand for the second form:
command |& tee combined.logKeep diagnostics separate without losing failures
Keeping stderr out of a data pipe preserves clean input for the next command, but a visible warning is not the same thing as a failing pipeline status. In a POSIX shell, a pipeline normally reports the status of its last command, so an upstream failure can be masked when the final consumer succeeds.
Bash and several other shells provide pipefail so the pipeline fails when one of its commands fails. Bash also exposes the individual statuses through PIPESTATUS immediately after the pipeline. These controls preserve failure information without mixing diagnostic text into the data stream.
- Do not merge stderr merely to make upstream failures visible to automation; status and data are different channels.
- Enable pipefail deliberately in scripts whose shell supports it, and declare that shell explicitly.
- Read PIPESTATUS before another command overwrites it.
- If portability matters, structure the workflow so each important command’s status can be checked directly.
Keep data clean and retain upstream failure in Bash
set -o pipefail
producer | consumer > result.txt
status=$?
printf 'pipeline status: %s\n' "$status"
# For per-command status, inspect immediately in Bash:
producer | consumer
printf 'producer=%s consumer=%s\n' "${PIPESTATUS[0]}" "${PIPESTATUS[1]}"Read redirections from left to right
A redirection changes a descriptor or duplicates its current target. In >out 2>&1, the shell first connects stdout to out, then makes stderr refer to stdout’s current target, so both enter the file.
In 2>&1 >out, stderr first copies stdout’s original target—often the terminal—then stdout moves to out. The two descriptors end at different destinations. The notation 2>&1 duplicates a descriptor relationship at that moment; it does not create a permanent link that follows future changes.
Compare the two orders
sh -c 'printf "out\n"; printf "err\n" >&2' >combined.log 2>&1
sh -c 'printf "out\n"; printf "err\n" >&2' 2>&1 >stdout-only.log
printf '%s\n' '--- combined.log'
cat combined.log
printf '%s\n' '--- stdout-only.log'
cat stdout-only.logRedirect only the stream the task requires
The shell can open a file on stdin with <, open one on stdout with >, and open one on stderr with 2>. Keeping stdout and stderr separate is especially useful in automation because a result file stays parseable while a diagnostic file preserves the reason for failure.
/dev/null is a device that discards writes and immediately reports end-of-file on reads. Suppressing an expected warning can be reasonable, but discarding stderr broadly can hide permission errors, partial processing, or resource exhaustion.
- Use < when a program expects bytes from standard input but you have a file.
- Use 2>error.log when diagnostics must be retained without contaminating stdout.
- Capture the exit status immediately after the command you are checking.
- Do not send errors to /dev/null until you know which failures it can conceal.
Keep data and diagnostics separate
sort < raw-names.txt > sorted-names.txt 2> sort-errors.log
status=$?
printf 'sort exit status: %s\n' "$status"
# Suppress only a deliberately understood diagnostic stream:
command-that-may-warn 2>/dev/nullDebug the descriptor graph, not just the command text
When a pipeline hangs or output disappears, ask what resource each standard descriptor points to in every process. A writer can block when a pipe reader is slow; a reader can wait forever when some process still holds a write end open; output can seem missing because a program buffers differently when stdout is not a terminal.
On Linux, inspect /proc/PID/fd while the process is running. For deeper debugging, a system-call tracer can show open, dup2, read, write, and close operations, but tracing production processes can expose data and add overhead, so follow your operational controls.
- 1Reduce the pipeline to two commands and verify each command alone.
- 2Identify the intended source for descriptor 0 and destinations for 1 and 2.
- 3Check redirection order and whether stderr was intentionally merged.
- 4Inspect live descriptor targets and process state when the pipeline remains stuck.
- 5Check every relevant exit status and validate the produced data.
Inspect a known Linux process without modifying it
pid=1234
ls -l "/proc/$pid/fd/0" "/proc/$pid/fd/1" "/proc/$pid/fd/2"
cat "/proc/$pid/fdinfo/1"Common questions
Frequently asked questions
Why does grep miss error messages from the command before it?
A normal pipe sends only the left command’s stdout to grep. Its stderr retains the previous destination, usually the terminal. Add 2>&1 before the pipe only if grep should receive diagnostics too.
Why can a failed command still produce a successful pipeline?
POSIX shells normally report the status of the pipeline’s last command, so a successful grep or tee can mask an upstream failure. In Bash, enable pipefail or inspect PIPESTATUS immediately; otherwise structure the script to check each important command directly.
What is the difference between 2>&1 >file and >file 2>&1?
Redirections apply left to right. The first form copies stderr to stdout’s old destination before stdout moves to the file; the second moves stdout first and then copies stderr to that file destination.
Keep learning
Sources and next steps
Related mental models
Standard input, output, and error
A command-line process normally starts with three conventional open streams: standard input for incoming data, standard output for primary results, and standard error for diagnostics.
Redirection
Redirection is shell syntax that opens, closes, or duplicates file descriptors for a command before that command begins executing.
Pipelines
A pipeline connects the standard output of each command to the standard input of the next, allowing multiple processes to transform a byte stream concurrently.