HTTP Methods and Making API Requests with cURL

HTTP Methods and Making API Requests with cURL

Ahoy, operator. Congratulations on gaining acceptance into the Arcane Academy of API Arts. I’m Professor Hendrik van Hoek, head of House Von Neumann. In House Von Neumann, we believe that code is infinite. We love solving problems of computer science and architecting new software. To do that, students of House Von Neumann must study the foundations of APIs.

You are probably familiar with the term HTTP from seeing websites and links, but what exactly is this elusive term?

HTTP (Hypertext Transfer Protocol) is a protocol that allows for communication between a client and server – it’s the language of the world wide web and is what allows you to access everything on the internet.

Here’s a quick (and oversimplified) glance at how you just used HTTP to arrive at this blog post:

Whether you approached this article through Google, a direct link, or the Twilio blog, an HTTP request was sent to the Twilio server by your browser.

This request consisted of multiple components including a method type and a request URL. For this request, the GET method was used along with the request URL,www.twilio.com/blog/http-methods-requests-curl.

This can usually be seen in the address bar of your browser. The first part of the URL, www.twilio.com, is the host URL and indicates the direction in which the server will send the request. The second part of the URL, /blog/http-methods-requests-curl, is the path. This tells the server which resource is being requested.

The server (Twilio) will receive the request and fetch the resource (this blog post) at the given path on the server. This resource is then sent within an HTTP response to your browser and will be displayed on your screen.

HTTP request methods

The GET request method is just one way to communicate to servers. Other possible requests one might have could be to send, replace, modify, or delete a resource on a server. To carry out these actions, there are methods such as  POST, PUT,  PATCH, and DELETE. These HTTP Request Methods specify to the server what type of action to perform on a resource.

The next few sections will elaborate on each of the main request methods and when they’re used. To better explain their usage, we’ll also show how they can be used on an e-commerce based API.

GET

The most common and widely used HTTP request type is the GET method – it’s used to retrieve a resource at a given path from the server. GET methods are considered idempotent. This means that regardless of how many times you execute the same request, the action, data, and response will always be the same.

For example, say you’re connecting to an e-commerce API and want to fetch all the products listed. Since you are just retrieving data and not modifying the product, you’d use the GET method and it might look something like this:

What if you want to fetch a particular product? Every resource that can be fetched has some unique identifier which can be specified in the URL path to retrieve the resource. Assuming the unique identifier for a product is an ID, the GET method to retrieve a particular product would look something like this:

POST

ThePOST request is used to send data to a server which can then create or update a resource. An example of when POST requests are used is whenever you fill out a form on a website and hit the submit button. The form data you entered is placed in the request body and then processed by the server. POST requests are non-idempotent since sending the same POST requests will create multiple resources with different unique identifiers making each resource new and unique.

Take a look at how this can be used on the same fictional e-commerce API from above. If you need to create a new product resource, you’d typically use the same endpoint that’s used to fetch the products but with a POST request:

To fill in the product details, the request body can be used to pass in data which will then be processed by the server:

JSON (used above) is one of the many data type formats that can be used in request bodies – other data types include XML, form data, and plain text.

Once the server receives this data, it will be processed, and a new resource will be created and appended to the/products endpoint with a unique ID. The new resource, along with its created ID, will be sent back to the client as a response.

PUT

PUT requests are fairly similar to the POST requests in that they are used to send data to a server, but only to update and replace a target resource. If the target resource is not present, a new resource will be created with the given data. This method overwrites a target resource with a payload which means sending the same PUT request will always produce the same result, making it idempotent.

Say there's an existing product with an ID of 272. Product 272 has empty data fields that need to be filled out:

To do this, you could use a PUT request. Using the ID of the target product, the PUT request and body would look like the following:

Request:

Request body:

The server will make a request to the target path and replace the resource with the processed data from the request body.

PATCH

PATCH is similar to PUT in that it updates a target resource on a server but is used to partially modify it rather than replacing it. Depending on the server or the API, PATCH requests can be idempotent but are technically non-idempotent since they can serve as functions and applying the same request could result in a different effect.

For a non-idempotent example, an API can implement a discounting function for a product where the request line and body is in the form of:

Request:

Request body:

Repeatedly applying this PATCH request to a product will continuously discount the price of the product.

For an idempotent example, say a product is on final sale and needs to be flagged as non-refundable. Using the ID of the target product, the PATCH request and body would look like the following:

Request:

Request body:

DELETE

As the name suggests, the DELETE request is used to delete a target resource. DELETE requests are idempotent since repeated requests return the same result: a deleted resource.

To delete a particular product from the example e-commerce API, the request line would look like the following:

HTTP Requests with cURL

Now that you’ve gone through the main HTTP methods, it’s time to  see them in action. To do this you’ll be using cURL to send out HTTP requests directly from your shell terminal. cURL is a popular command line tool for transferring data with URL through protocols including HTTP, SMTP, TELNET and more.

To ensure cURL is installed on your system, open your terminal and enter curl --version to see if a version number shows up. If not, you can download and install it from cURL’s website: https://curl.se/dlwiz/.

GET request with NewsAPI

To demonstrate a simple HTTP GET request, you’ll be using the NewsAPI to fetch today's top headlines. To get started, head over to the register page on NewsAPI to sign up and get your API key.

Once you’ve registered for an API key, navigate to your terminal and paste in the following, taking care to replace the placeholder API key with your new API key:

Before you hit enter, explore the different parts of this command and the provided URL:

• The -X option is used to specify which HTTP method will be used for the request followed by the method type, GET.
• The first part of the URL, http://newsapi.org, specifies the host of the server and where the request is being sent to.
• The next part of the URL, /v2/top-headlines, is the path and it specifies to grab the /top-headlines resource.
• The last part of the URL, ?country=us&pageSize=2&apiKey=XXXX, includes the request parameters which define values that are used to process the request on the server.

The parameters included in the URL tells the server to grab top headlines from the country us, and limit the size of the page to 2. The apiKey parameter is used to authorize your request so like mentioned above, don’t forget to replace the XXXX with your registered API key.

If you’d like to play around with the request parameters or know more about them for this particular endpoint, check out the NewsAPI documentation on it: Endpoints - Top Headlines. Reading an API’s documentation is the best way to understand how it works.

Enter the command and your output will show the top two headlines in the US for today. Your HTTP response, specifically the articles shown, will differ depending on the date, but it should look something like this:

POST request with Twilio API

Now that you’ve seen how to make an HTTP GET request, let’s take it a step further and demonstrate how to make a POST request with an included request body using cURL.

You’ll be using Twilio’s Programmable Messaging API to send a text message to yourself all through an HTTP request. To supplement this demonstration, you’ll be following along with this Twilio Blog tutorial: How to Send an SMS from a Shell Script using cURL. 

Follow the tutorial to set up your Twilio account and phone number. Head over to the Twilio Console and grab your Account SID, Auth Token, and your Twilio phone number from the red box shown below:

In your terminal, define your Twilio credentials as environment variables:

The TO_NUMBER will be the number that your text will be sent to.

If you are following this tutorial on a Windows computer, use set instead of export to define your environment variables in the command prompt. For the phone numbers, use the E.164 format, which includes a plus sign and the country code.

Using the environment variables in shell commands will differ based on your operating system. If you are a using Unix or MacOS, the command to send out an SMS to your number will be the following:

If your operating system is Windows, the command will be the following:

The only difference between these two commands is the syntax of the environment variables.

Here’s how to decode this command:

• The -X option followed by POST specifies the type of HTTP request
• The -d option followed by a string specifies the data that is gonna be passed in the request body of the POST request. To send an SMS with the Twilio API through an HTTP request, three data inputs are required:

• Body - The text message that will be sent
• From - The Twilio number that will be used to send the SMS
• To - The phone number where the SMS will be sent to

• The -u option followed by a string specifies the username and password for authentication to a server. In this case, your Twilio Account SID and Auth Token will be used as the username and password respectively.
• The last parameter for the command is the URL of where the request will be sent to. The API endpoint being targeted is /Messages.

As previously stated, sending a POST request will send data to a server which can then create or update the given resource. In this case, sending a POST request to the /Messages endpoint will result in creating a Message resource which represents an inbound or outbound message.

Enter the command and in a moment you’ll receive an SMS from your Twilio number along with the following HTTP response in your terminal:

Conclusion

Congratulations, operator! You’ve just learned about the basics of the HTTP protocol, methods and requests, and you got the chance to experiment and make HTTP requests with cURL.

When it’s time to be placed into a House at the Academy, I hope you’ll channel what you’ve learned here and embrace the magic of House Von Neumann.

Download TwilioQuest to begin your studies immediately!

Dhruv Patel is a Developer on Twilio’s Developer Voices team. You can find Dhruv working in a coffee shop with a glass of cold brew or he can either be reached at dhrpatel [at] twilio.com or LinkedIn.