How does curl print to terminal while piping

Question

When I curl a file, and pipe it to a file or another command, I see output in my terminal. I am not sure how this happens, as the pipe is supposed to take all the output from curl, right?

For example:

$ curl http://www.archive.org/stream/Pi_to_100000000_places/pi.txt > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  129M    0  129M    0     0  22.5M      0 --:--:--  0:00:05 --:--:-- 24.7M

---

Edit

I use curl like this:

curl http://www.archive.org/stream/Pi_to_100000000_places/pi.txt | some_other_command > some_file

I do not want to pipe the status to some_other_command, I was just wondering how it was able to display the status. However, showing how to redirect both streams added to the answer, so don't remove that.

Answer 1

There are two output streams generally available: standard output, and standard error. In practice, when running in a terminal, both send data to the terminal. > only redirects standard output, and curl prints the progress data to standard error. To suppress both, use one of:

curl ... > /dev/null 2>&1
curl ... &> /dev/null    # bash's combined redirection operator
curl -s ...    # -s, --silent: Silent or quiet mode. Don't show progress meter or error messages.

To send both to a pipe:

curl ... 2>&1 | ...
curl |& ...    # bash's combined pipe

Unless you use the |& or &> operators, all streams are redirected independently.

Also see:

• Stack Overflow: confused about stdin, stdout and stderr?

Answer 2

When you are using curl to open an URL, you'll get two output:

1. The status of the curl itself.
2. The contents of that URL.

Curl should use a way to show these two separately otherwise processing of the real output (URL's content) would be hard and I'll end up with unnecessary contents (curl's status).

So it uses stderr for its status and stdout for the content.

Using > you are redirecting the URL's content (stdout) to the /dev/null, you should actually use: 2> /dev/null instead.

Also if you want to pip both of them to the next command:

curl url |&  command

If you only want the content be piped to next command while not seeing the status:

curl 2> /dev/null | command