Table of Contents

1. Challenge

   Introduction

   Welcome to the lab

   Guided: Creating HTTP Requests in Python

   Introduction

   As you develop Python applications, you'll often need to interact with external services and APIs.

   REST APIs have become the standard way for applications to communicate, enabling data exchange through HTTP requests.

   This hands-on lab demonstrates how to use Python's requests library - a popular tool that simplifies working with APIs. This library offers a clean interface for sending HTTP requests and handling responses, with simpler syntax than Python's built-in urllib library.

   The requests library has already been installed in this lab, but if you'd like to explore the requests library in your own environment, simply install it with:

   pip install requests
   

   In the following steps, you'll learn to:

   • Make HTTP requests (GET, POST)
   • Process various response formats (text, JSON)
   • Handle errors effectively
   • Manage authentication and sessions

   Throughout this lab, you'll use a local version of HTTPBin as your testing endpoint - a service designed specifically for HTTP request testing that provides consistent, predictable responses for all request types covered in this lab.

   If you get stuck with any of the steps, refer to the /solution directory provided.

   To begin, click the right-arrow below.

2. Challenge

   Making a Basic GET Request

   Making a Basic GET Request

   In this step, you'll make a simple GET request using Python's requests library to retrieve the /get endpoint from HTTPBin.

   The response will include details about the request, such as headers, arguments, and origin.

   Sending a GET Request

   To send a basic GET request, you can use the requests.get() method.

   Open the **/get.py
   ** script file, and add the following code after the import statements.

   response = requests.get("http://localhost:3000/get")
   
   print("Status Code:", response.status_code)
   

   This code sends a GET request to http://localhost:3000/get and saves the response in the response variable, which holds information like the status code, headers, and body of the response.

   Running the Script

   To run this script, execute the following command in the terminal:

   In the terminal, you should see the Status Code: 200 for a successful request.

   Reviewing the Headers

   The response object also includes metadata about the server's response, stored in a dictionary called headers. This dictionary provides details such as the content type, content length, and server information.

   You can review this information by adding the following code to the get.py script:

   print("Headers:") 
   pprint(dict(response.headers))
   

   Notice how the pprint() function, combined with casting the headers to a dictionary, makes the output more readable.

   Rerunning this script in the terminal:

   Reviewing the headers, you'll see the Content-Type header, which indicates the type of data returned by the server. The /get endpoint of the HTTPBin server returns a JSON response, so the Content-Type header should be application/json.

   In the next step, you will learn how to retrieve the content of the response.

3. Challenge

   Parsing the Response Body

   Parsing the Response Body

   Up to this point, your get.py script checks the response's status code and headers. Now it's time to access and work with the response body—the actual data returned from the server.

   The requests library provides a few ways to read the response content.

   .text vs .json()

   Two of those options are .text and .json().

   You could use response.text to retrieve the raw response body as a string. However, since the Content-Type header indicates that the content is of type: application/json, you can utilize the response.json() method to directly parse the body into a Python dictionary, making it more convenient to work with.

   To see this you can add the following lines to your get.py script:

   data = response.json()
   pprint(data)
   

   By storing the json() response in the data variable, the content body is converted into a dictionary. This allows pprint() to format and display the dictionary in a readable and organized manner.

   Rerunning the get.py script you can see this parsed JSON output:

   python3 get.py
   

   Retrieving specific values

   You can extract specific values from the parsed JSON by accessing them with Python's dictionary syntax, like data['key'].

   Append the following line to your get.py script to retrieve the URL from the parsed JSON:

   print(data['url'])
   

   Rerun the script to see this specific value:

   python3 get.py
   

   In the next step, you'll learn how to handle potential errors when parsing the response body to ensure your script runs smoothly.

4. Challenge

   Handling Errors

   Handling Errors

   When working with HTTP requests, errors can occur due to various reasons. Following are some steps you can use to handle common errors using Python's requests library:

   The main thing you'll notice is that the requests library has a number of exceptions that you can catch to handle different types of errors.

   Generic Request Exceptions

   The first exception to consider is requests.exceptions. RequestException. As the base exception for all request-related errors, it serves as a general-purpose handler for any issues that may arise during an HTTP request, making it an ideal starting point for generic error handling.

   Opening the **/errors/connection.py ** file. Notice that the get() request is sent to the non-existent :3001 port, which is unavailable.

   Also, the try block is generically catching the base Exception class.

   Your task is to update the code to first catch requests.exceptions. RequestException, as it is more specific and tailored to handle errors related to HTTP requests. This ensures that HTTP-related exceptions are handled before falling back to the more generic Exception class.

   Refactor the script to match the following code snippet:

   try:
       response = requests.get("http://localhost:3001/get")
       print("Should not be reached", response)
   except requests.exceptions.RequestException as e:
       print(f"A RequestException occurred: {e}")
   except Exception as e:
     print(f"An error occurred: {e}")
   

   You can now run this script file to see the caught RequestException.

   In the terminal, you should see the output A RequestException occurred: . . ., indicating that you have successfully caught a base exception related to HTTP requests.

   Handling Connection Errors

   Examining the terminal output, you can observe that the exception specifically indicates Connection refused.

   When a connection error occurs, it is essential to handle it gracefully. The requests.exceptions. ConnectionError exception is specifically designed to handle connection-related issues, such as when the server is unreachable or the connection is refused.

   Refactor the script to first catch the more specific requests.exceptions. ConnectionError exception and print a custom message when this error occurs:

   try:
       response = requests.get("http://localhost:3001/get")
       print("Should not be reached", response)
   except requests.exceptions.ConnectionError as e:
       print(f"A ConnectionError occurred: {e}")
   ...
   

   Rerun the script using the command:

   python3 errors/connection.py
   

   You should see the output A ConnectionError occurred: . . ., indicating that the specific connection error was successfully caught and handled.

   Handling HTTP Errors

   Sometimes, the server may return an HTTP error status code (4xx or 5xx). By default, the requests library does not treat these as exceptions.

   This can be seen by opening the /errors/httperrors.py file.

   The HTTPBin server is used to simulate a 500 server error by sending a request to the /status/500 endpoint.

   Notice that the requests.exceptions. HTTPError exception is being captured in the except block.

   But, when running this script file using the command:

   python3 errors/httperrors.py
   

   You should see that the script is printing the Should not be reached comment instead of the expected An HTTPError occurred.

   This demonstrates that the 500 server error returned by the HTTPBin server was not automatically raised as an exception by the requests library. This highlights the importance of explicitly calling response.raise_for_status() to raise HTTP error responses as a requests.exceptions. HTTPError exception.

   Refactoring the script, insert a call to response.raise_for_status() method after the .get() request.

   try:
       response = requests.get("http://localhost:3000/status/500")
       response.raise_for_status()
   except requests.exceptions.HTTPError as e:
       print(f"An HTTPError occurred: {e}")
   

   And rerunning the script:

   python3 errors/httperrors.py
   

   You should now see the output An HTTPError occurred: . . .

   By understanding and handling these common exceptions, you can ensure that your application gracefully manages errors during HTTP requests.

   Now that you've learned how to handle errors effectively, the next step is to explore how to send POST requests to interact with APIs and submit data.

   Click the next step, to explore how to send POST requests to interact with APIs and submit data.

5. Challenge

   Sending POST Requests

   Sending POST Requests

   When working with REST APIs, POST requests are used to send data to the server. The requests library makes it easy to send POST requests with different types of data.

   To test this functionality, the HTTPBin /post endpoint is used because it echoes back the request details, including headers, form data, and JSON payloads. In the following tasks, the response will be printed to display what the server received from the request.

   Before jumping in, one thing you'll notice in the following examples is that both versions use a Python dictionary to pass the data to be sent in the POST request. This makes it easy to work with structured data in your POST requests.

   Sending Form Data

   To send form-encoded data, use the data= parameter of the requests.post() method. This is commonly used when submitting form data to an endpoint, such as when interacting with web forms or APIs that expect application/x-www-form-urlencoded content type.

   Open the **/data/formdata.py ** file. Notice the form_data dictionary has been created with key-value pairs that represent form data. Note that this data is similar to what you would see in an HTML form submission.

   Use the data= parameter to create the following post() request and print the HTTPBin echoed response to the terminal.

   response = requests.post(url, data=form_data)
   
   data = response.json()
   pprint(data)
   

   Run the script in the terminal using the command:

   python3 data/formdata.py
   

   Here are a few key points to observe in the echoed response from the HTTPBin server:

   • The form key displays the form data that was sent in the POST request.
   • The headers key shows the request headers, including the Content-Type header, which indicates that the data sent in the request was of the application/x-www-form-urlencoded type.

   Sending JSON Data

   While form data is useful for simple key-value pairs and is commonly used in web forms, it is limited to a single level and cannot handle hierarchical or nested data structures.

   JSON, on the other hand, is preferred for REST APIs due to its structured format, support for nested data, and widespread adoption, making it ideal for complex data and modern web services.

   To send JSON data, use the json= parameter in the requests.post() method. This is useful when interacting with APIs that expect JSON payloads.

   In the **/data/jsondata.py ** file, use the json= parameter to send the json_data in a post() request. Print the HTTPBin echoed response to the terminal.

   Append the following code to the script:

   response = requests.post(url, json=json_data)
   data = response.json()
   pprint(data)
   

   Rerunning the script:

   python3 data/jsondata.py
   

   Observe the following in the echoed response from the HTTPBin server, :

   • The json key displays the JSON data that was sent in the POST request.
   • The headers key shows the request headers, including the Content-Type header, which indicates that the data sent in the request was of the application/json type.

   In the next step, you'll learn how to manage sessions and reuse authentication tokens across multiple API calls.

6. Challenge

   Managing Sessions

   Managing Sessions

   When working with APIs, you may need to maintain state across multiple requests. The requests.Session object in Python makes it easy to manage such state.

   Why Use Sessions?

   Using a session allows you to:

   • Persist cookies across requests.
   • Reuse headers or authentication tokens.
   • Improve performance by reusing the underlying TCP connection.

   Demonstrating Sessions with Cookies

   To demonstrate the use of sessions, you can utilize the HTTPBin endpoints /cookies and /cookies/set, which are specifically designed for testing cookie management.

   The following example highlights how cookies are not preserved across separate requests calls but can be seamlessly maintained when using a Session.

   Opening the /session.py file, notice that the script is currently making a simple .get() request to the HTTPBin /cookies/set and /cookies endpoints.

   Running this script in the terminal using the command:

   Observe that the second request to /cookies, which returns any cookies stored in the request, does not include any cookies. This happens because the cookies set in the first request are not retained across separate requests.

   Using a Session Object

   This can be resolved by using a session object to manage the cookies.

   Below the # Create a session object comment, instantiate a Session object and refactor the requests.get() calls to utilize the newly created session object instead.

   The refactored script should now look like:

   # Create a session object
   session = requests.Session()
   
   session.get(set_cookie_url)
   response = session.get(get_cookies_url)
   

   Rerunning this script, you'll see that the cookies set in the first get() call are now persisted across session requests.

   By using sessions, you can simplify your code and make your API interactions more efficient.

   Closing the Session

   While this script demonstrates how to use a session to persist cookies across requests, it is crucial to remember that in a real-world application, the session must be properly closed to release the underlying resources, such as open network connections.

   You can explicitly close a session by calling session.close() method. Alternatively, you can use a with block to ensure the session is automatically closed when the block is exited.

   For example you could refactor the script to use a with statement as follows:

   with requests.Session() as session:
       session.get(set_cookie_url)
       response = session.get(get_cookies_url)
       data = response.json()
       print("Cookies:", data)
   

   This ensures that the session is closed when the block is exited, releasing any resources associated with the session.

   You can rerun the script to see that the cookies are still persisted across requests.

   python3 session.py
   

   Congratulations

   Well done on completing the lab!

   You have gained valuable knowledge on utilizing the requests library. With these skills, you are now better prepared to work with APIs efficiently.