the_information_nexus/tech_docs/python/requests.md

The requests library in Python is a simple, yet powerful HTTP library, designed for human beings. It provides methods to send all kinds of HTTP requests, making it invaluable for a wide range of applications involving HTTP requests to interact with web services or APIs.

Requests Library Usage Guide

Installation

pip install requests

Basic HTTP Requests

The core of requests is its ability to make all types of HTTP requests. Here are brief examples categorized by request type:

GET Requests

Retrieve data from a specified resource.

import requests

response = requests.get('https://api.github.com')
print(response.json())  # Convert JSON response to a dictionary

POST Requests

Send data to a server to create/update a resource.

payload = {'key': 'value'}
response = requests.post('https://httpbin.org/post', data=payload)
print(response.text)

Working with Parameters and Headers

Passing URL Parameters

Send GET requests with query parameters.

params = {'q': 'Python HTTP library'}
response = requests.get('https://www.google.com/search', params=params)
print(response.url)  # Check the formed URL

Custom Headers

Modify request headers for tasks like setting user-agent or authentication tokens.

headers = {'User-Agent': 'My App/0.0.1'}
response = requests.get('https://api.github.com', headers=headers)

Handling Response Content

Response Status Codes

Check response status to handle different scenarios like success or error.

if response.status_code == 200:
    print('Success!')
elif response.status_code == 404:
    print('Not Found.')

JSON Response Content

Directly access JSON response content for APIs that send back JSON data.

response = requests.get('https://api.github.com')
print(response.json())  # JSON response content

Advanced Usage

Sessions with Context Managers

Use Sessions for keeping certain parameters across requests.

with requests.Session() as session:
    session.headers.update({'User-Agent': 'My App'})
    response = session.get('https://api.github.com')

Handling Timeouts

Specify a timeout duration to avoid hanging indefinitely.

try:
    response = requests.get('https://api.github.com', timeout=1)
except requests.Timeout:
    print('The request timed out')

Project Ideas

• Web Scraping: Automate data collection from websites, respecting their robots.txt policies, to gather data for analysis.

• API Consumption: Develop applications that interact with various third-party APIs for tasks like social media posting, weather updates, or cryptocurrency price tracking.

• Automated Testing: Create scripts to test web applications' APIs, checking their responses, status codes, and performance.

• Website Monitoring: Develop a tool to monitor website uptime by periodically sending requests and alerting you via email or SMS when your site goes down.

Security and Best Practices

• Sensitive Data: Avoid sending sensitive information like passwords or API keys as URL parameters. Use headers or POST requests instead.
• Error Handling: Always check for response status codes and handle exceptions like connection errors or timeouts gracefully.
• Session Objects: Use Session objects for persistent connections, especially when making multiple requests to the same host.

The requests library