The curl command-line tool is one of the most useful and versatile programs you’re able to learn. Its versatility and comprehensive HTTP implementation mean that, if there’s a URL for it, curl can do it.

8Fetch a Web Page

In its most basic form,curlfetches a URL and displays its contents on your terminal:

What you see will depend on the content type of the URL. If you request a web page, you’ll see its HTML source:

The output from a simple curl request showing HTML source.

You canredirect this output to a fileto save a web page:

Notice how curl detects that you’re redirecting the normal output and sends timing information to stderr:

A curl command with output redirected to a file, showing timing information for the transfer.

A URL may contain characters with special meaning to the shell, like “?” or “#.” You’ll need to be careful to avoid problems this may cause, so verify you surround URLs with single quotes if they contain special characters. If you’re not sure, be cautious and always quote them:

If you try to use curl to download a binary file—like an image or a Microsoft Word document—you’ll see an error telling you that “Binary output can mess up your terminal.” This is because curl will print the response body that it receives to output. In the case of an image, this is almost certainly not what you want to be doing.

Output from curl showing two forms of file download, the latter with a simple one-line progress bar.

The error message mentions the –output option, which lets you specify a file to save the output to rather than print it. you’re able to also use -o as a shorter equivalent:

This will save the file at that URL in the current directory. As before, since curl is no longer outputting the response, it provides a lot of detail about the time your download takes. If you want a simple progress bar instead, use the-#or–progress-baroption:

Output from a curl request to ifconfig.co shows an IP address.

6Install Software

You may sometimes come across installation instructions that ask you to use curl. For example,atuin—a replacement forthe history command—has the following installation instruction:

If you open that URL in a web browser, you’ll see a shell script that performs installation and setup. You can download it and run it manually if you want, the shell one-liner is just a quicker way. Note that the output—the contents of this script—is piped to sh which will run it as a shell script.

The ifconfig.co home page in a web browser with details including the user’s public IP address. their country, and their region.

The suggested curl usage in the atuin example uses these options:

Running a shell script downloaded from the internet is risky because you don’t know exactly what it will do. Not only do you need to trust the source, but you also need to be confident you’re secure from anyMan-in-the-Middle attacks. HTTPS can help to mitigate this risk, but if you want to be extra careful, consider dropping the pipe, downloading the file, and checking that it’s safe to run before you do so.

Output from a curl request to ifconfig.co with a modified user agent showing HTML source in response.

5Check Your IP Address

This practical use for curl is a quick and easy way ofdiscovering your public IP address, thanks to the service at ifconfig.co:

The site responds with your public IP address:

This default output is plain text which makes it an excellent resource for automation. You might notice that you get a very different result if you go to the same URL in your web browser:

This is because the site behaves differently based on your user agent. By default, curl sends something like “curl/8.7.1” in the User-Agent header. If you ever need to set this to something else, e.g. for testing purposes, use the -A/–user-agent option:

A curl command showing response headers including Content-Type and Last-Modified..

Now the ifconfig site will respond with HTML, just as it does in a web browser:

4Show Response or Request Headers

Sometimes, you want to fetch just the response headers from a URL. You might be debugging an error or checking cache headers to see if you need to redownload a file. To view just the response headers, use -I/–head:

This will send a HEAD request, which not all web servers support. If you get an error using this method, you’re able to tell curl to send a GET request and only output the headers:

Output from curl with the –verbose and -I options to show both response and request headers.

If you want to see the request headers that curl sends, use the –verbose/-v option:

3Access an API

If you’re working with an API, a good client likePostmanorPostingwill help you automate and debug the task. But curl can be just as useful for exploring how an API works.

TheDummyJSONservice is a good place to start. This site provides an example API with dummy data that you’re able to use to test client apps and your own programs. You can use different endpoints to check redirects, test authorization, and simulate time delays or random responses.

JSON-formatted output from a curl request to dummyjson.com.

Trying out the service with curl is as simple as passing the appropriate endpoint:

You should see a simple JSON document in response, with two properties. The method property will reflect the HTTP method of your request:

JSON-formatted data representing a product obtained from dumyjson.com using curl and jq.

Once you’ve fetched data from an API, you can easily process it using standard command-line tools. For example, you canprocess JSON data using jq. This pipeline fetches product data using curl, then passes it to jq to extract the first product:

2Fetch an RSS Feed

One of my current projects is a shell-based RSS client. At its core lies curl because the tool is so convenient and supports every low-level HTTP operation I need. For example, you may fetch anRSS feedand examine its contents using curl:

If you have libxml installed, you’re able to process the XML output using xmllint:

This pipeline uses xmllint to evaluate an XPath expression that extracts data from XML, in this case, the title of the channel:

1Use curl Programmatically

The cURL project is about more than just curl, the command-line tool. It also includes a library, libcurl, that curl uses itself. This library is highly portable and available for all manner of operating systems including Linux, Windows, macOS, and Android. libcurl comes with a C API, but there arebindingsfor many other languages like C++, Python, Rust, and Go.

The library mirrors much of what the command-line tool does, particularly its options and their behavior. Here’s a basic example of PHP usage:

The curl_init() function starts a session and returns a handle that you can use to access it. You can control the library’s behavior using the options that curl supports via curl_setopt(). curl_exec() actually sends the request while curl_error() and curl_close() are useful admin functions.

This example uses the CURLOPT_FILE setting to write the response body to an open file handle. Note that this is the equivalent of curl’s –output option.

If you’re looking for other terminal-based tools to master, learnsome basic git commandsor look atsome bash script examples.