How to Simulate a Browser Visit Using Python's Requests: How can I make my Python requests look like they're coming from a real browser?

How to Simulate a Browser Visit Using Python's Requests: A Guide to Faking User Agents

When attempting to retrieve web content using Python's Requests or wget, you may encounter unexpected results compared to using a standard browser. This is because websites often implement protections to prevent automated queries. To overcome this challenge, you can fake a browser visit by providing a User-Agent header.

Implementing the User-Agent Header

To fake a browser visit, you need to include a User-Agent header with your request. This header specifies the type of browser and device used, making your request appear more like a legitimate user. Here's an example using Python's Requests:

import requests

# Define the target website URL
url = 'http://www.ichangtou.com/#company:data_000008.html'

# Create a dictionary of headers with a valid User-Agent string
headers = {'User-Agent': 'Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/39.0.2171.95 Safari/537.36'}

# Send the request with the User-Agent header
response = requests.get(url, headers=headers)

# Print the response content
print(response.content)

Additional Resources

• For a complete list of User-Agent strings, refer to [this resource](https://deviceatlas.com/blog/list-of-user-agent-strings).
• For more advanced user agent fakery, consider using the third-party package [fake-useragent](https://pypi.org/project/fake-useragent/).

The above is the detailed content of How to Simulate a Browser Visit Using Python's Requests: How can I make my Python requests look like they're coming from a real browser?. For more information, please follow other related articles on the PHP Chinese website!