5

When I curl a file, and pipe it to a file or another command, I see output in my terminal. I am not sure how this happens, as the pipe is supposed to take all the output from curl, right?

For example:

$ curl http://www.archive.org/stream/Pi_to_100000000_places/pi.txt > /dev/null
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  129M    0  129M    0     0  22.5M      0 --:--:--  0:00:05 --:--:-- 24.7M

Edit

I use curl like this:

curl http://www.archive.org/stream/Pi_to_100000000_places/pi.txt | some_other_command > some_file

I do not want to pipe the status to some_other_command, I was just wondering how it was able to display the status. However, showing how to redirect both streams added to the answer, so don't remove that.

  • See superuser.com/q/173209/53494. – edwinksl Aug 10 '17 at 2:33
  • You're redirecting curl's output to /dev/null for testing, right? I'm asking because the answers so far all feature how to redirect its standard output and standard error streams together. But if you're using curl to download files on standard output, you do not want to redirect stdout and stderr to the same place--any output on stderr would corrupt the download! I know you've already accepted an answer, but would you be willing to edit this with an example of how you're using curl? I think that would make this question more valuable to future readers in similar situations. – Eliah Kagan Aug 10 '17 at 15:53
  • @EliahKagan Sure, thanks for the suggestion. I am piping it to /dev/null just for demonstration – vikarjramun Aug 10 '17 at 18:33
  • Maybe you want to do this? echo curl -s http://api.ipify.org | some_command – Dino Douglas Mar 15 '18 at 15:25
8

There are two output streams generally available: standard output, and standard error. In practice, when running in a terminal, both send data to the terminal. > only redirects standard output, and curl prints the progress data to standard error. To suppress both, use one of:

curl ... > /dev/null 2>&1
curl ... &> /dev/null    # bash's combined redirection operator
curl -s ...    # -s, --silent: Silent or quiet mode. Don't show progress meter or error messages.

To send both to a pipe:

curl ... 2>&1 | ...
curl |& ...    # bash's combined pipe

Unless you use the |& or &> operators, all streams are redirected independently.

Also see:

| improve this answer | |
  • Seems like this also works with &>> to output to a file. As mentioned without putting curl into silent mode then the contents include the progress data, but I actually want that in my case. – Daniel Black Jun 5 at 19:29
2

When you are using curl to open an URL, you'll get two output:

  1. The status of the curl itself.
  2. The contents of that URL.

Curl should use a way to show these two separately otherwise processing of the real output (URL's content) would be hard and I'll end up with unnecessary contents (curl's status).

So it uses stderr for its status and stdout for the content.

Using > you are redirecting the URL's content (stdout) to the /dev/null, you should actually use: 2> /dev/null instead.

Also if you want to pip both of them to the next command:

curl url |&  command

If you only want the content be piped to next command while not seeing the status:

curl 2> /dev/null | command
| improve this answer | |

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service, privacy policy and cookie policy

Not the answer you're looking for? Browse other questions tagged or ask your own question.