2

How can I use curl to fetch pages using brotli compression?

According to release notes this should be somehow possible to do. My curl version seems ok:

$ curl -V
curl 7.58.0 (x86_64-pc-linux-gnu) libcurl/7.58.0 OpenSSL/1.1.1 zlib/1.2.11 libidn2/2.0.4 libpsl/0.19.1 (+libidn2/2.0.4) nghttp2/1.30.0 librtmp/2.3
Release-Date: 2018-01-24
Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtmp rtsp smb smbs smtp smtps telnet tftp 
Features: AsynchDNS IDN IPv6 Largefile GSS-API Kerberos SPNEGO NTLM NTLM_WB SSL libz TLS-SRP HTTP2 UnixSockets HTTPS-proxy PSL

apt-get install also claims curl is already the newest version (7.58.0-2ubuntu3.8).

However when I'm trying to fetch a page I'm getting errors:

$ curl --compressed https://httpbin.org/brotli
curl: (61) Unrecognized content encoding type. libcurl understands deflate, gzip content encodings.
$ curl https://httpbin.org/brotli
Warning: Binary output can mess up your terminal. Use "--output -" to tell 
Warning: curl to output it to your terminal anyway, or consider "--output 
Warning: <FILE>" to save to a file.

What am I missing? I also tried to install libbrotli-dev, but this didn't change anything.

3
  • 1
    this is me right now imgs.xkcd.com/comics/wisdom_of_the_ancients.png Jun 18, 2020 at 8:22
  • @Integralist good to know :) in the end I switched to fetching these pages with a python script (requests, brotli packages)
    – pieca
    Jun 20, 2020 at 8:28
  • curl 7.71.1 (x86_64-redhat-linux-gnu) libcurl/7.71.1 OpenSSL/1.1.1k-fips zlib/1.2.11 brotli/1.0.9 libidn2/2.3.1 libpsl/0.21.1 (+libidn2/2.3.0) libssh/0.9.5/openssl/zlib nghttp2/1.43.0 Release-Date: 2020-07-01 Protocols: dict file ftp ftps gopher http https imap imaps ldap ldaps pop3 pop3s rtsp scp sftp smb smbs smtp smtps telnet tftp Features: AsynchDNS brotli GSS-API HTTP2 HTTPS-proxy IDN IPv6 Kerberos Largefile libz Metalink NTLM NTLM_WB PSL SPNEGO SSL TLS-SRP UnixSockets I have same issue even after brotli support. Any help? Jul 3, 2021 at 9:06

2 Answers 2

2

An option is to use standalone brotli tool to decompress the file, e.g.

curl https://httpbin.org/brotli | brotli -d -

1

You need to either wait until Ubuntu add libbrotli support to curl (maybe at Ubuntu 20) ... or recompile curl with brotli support.

"Recompile" may sound scary but actually is something rather easy, here you can find the instructions about it -> https://curl.haxx.se/docs/install.html (please ensure you add the new option --with-brotli and the libbrotli-dev package you mentioned above to work properly).

Your Answer

By clicking “Post Your Answer”, you agree to our terms of service and acknowledge that you have read and understand our privacy policy and code of conduct.

Not the answer you're looking for? Browse other questions tagged or ask your own question.