如何采集防采集的网站
我想用php采集一个网站的数据,但是无法获取该网站的数据。网址如下:
http://www.alldatasheet.com/view.jsp?Searchword=78HC
希望您能试一下,只要能返回数据就行了。我试了很久不能成功。
回复讨论(解决方案)
$header = array ( "GET /view.jsp?Searchword=78HC HTTP/1.1","Host: www.alldatasheet.com","Connection: keep-alive","Cache-Control: max-age=0","Accept: text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8","User-Agent: Mozilla/5.0 (Windows NT 6.1) AppleWebKit/537.22 (KHTML, like Gecko) Chrome/25.0.1364.152 Safari/537.22","Accept-Encoding: gzip,deflate,sdch","Accept-Language: en-US,zh-CN;q=0.8,zh;q=0.6","Accept-Charset: UTF-8,*;q=0.5","Cookie: JSESSIONID=BD1418BC3F4CA9084F0C022A98687A09; track_id=117.25.173.111363310326444; seekstr=*78H*..; seekshot=78H..1..75..8..112; __utma=191189370.2036196682.1363308553.1363308553.1363308553.1; __utmb=191189370.3.10.1363308553; __utmc=191189370; __utmz=191189370.1363308553.1.1.utmcsr=(direct)|utmccn=(direct)|utmcmd=(none); arp_scroll_position=900"); // 初始化一个 cURL 对象$curl = curl_init(); // 设置你需要抓取的URLcurl_setopt($curl, CURLOPT_URL, 'http://www.alldatasheet.com/view.jsp?Searchword=78HC');curl_setopt($curl, CURLOPT_HTTPHEADER, $header); //设置header // 设置header显示方式curl_setopt($curl, CURLOPT_HEADER, 0); // 设置cURL 参数,要求结果保存到字符串中还是输出到屏幕上。curl_setopt($curl, CURLOPT_RETURNTRANSFER, 1); // 运行cURL,请求网页$data = curl_exec($curl); // 关闭URL请求curl_close($curl); // 显示获得的数据var_dump($data);
只要是浏览器能访问的页面,应该都能采集的。
关键是cookie。

Hot AI Tools

Undresser.AI Undress
AI-powered app for creating realistic nude photos

AI Clothes Remover
Online AI tool for removing clothes from photos.

Undress AI Tool
Undress images for free

Clothoff.io
AI clothes remover

Video Face Swap
Swap faces in any video effortlessly with our completely free AI face swap tool!

Hot Article

Hot Tools

Notepad++7.3.1
Easy-to-use and free code editor

SublimeText3 Chinese version
Chinese version, very easy to use

Zend Studio 13.0.1
Powerful PHP integrated development environment

Dreamweaver CS6
Visual web development tools

SublimeText3 Mac version
God-level code editing software (SublimeText3)

Hot Topics

Alipay PHP...

JWT is an open standard based on JSON, used to securely transmit information between parties, mainly for identity authentication and information exchange. 1. JWT consists of three parts: Header, Payload and Signature. 2. The working principle of JWT includes three steps: generating JWT, verifying JWT and parsing Payload. 3. When using JWT for authentication in PHP, JWT can be generated and verified, and user role and permission information can be included in advanced usage. 4. Common errors include signature verification failure, token expiration, and payload oversized. Debugging skills include using debugging tools and logging. 5. Performance optimization and best practices include using appropriate signature algorithms, setting validity periods reasonably,

Session hijacking can be achieved through the following steps: 1. Obtain the session ID, 2. Use the session ID, 3. Keep the session active. The methods to prevent session hijacking in PHP include: 1. Use the session_regenerate_id() function to regenerate the session ID, 2. Store session data through the database, 3. Ensure that all session data is transmitted through HTTPS.

The application of SOLID principle in PHP development includes: 1. Single responsibility principle (SRP): Each class is responsible for only one function. 2. Open and close principle (OCP): Changes are achieved through extension rather than modification. 3. Lisch's Substitution Principle (LSP): Subclasses can replace base classes without affecting program accuracy. 4. Interface isolation principle (ISP): Use fine-grained interfaces to avoid dependencies and unused methods. 5. Dependency inversion principle (DIP): High and low-level modules rely on abstraction and are implemented through dependency injection.

How to debug CLI mode in PHPStorm? When developing with PHPStorm, sometimes we need to debug PHP in command line interface (CLI) mode...

How to automatically set the permissions of unixsocket after the system restarts. Every time the system restarts, we need to execute the following command to modify the permissions of unixsocket: sudo...

Static binding (static::) implements late static binding (LSB) in PHP, allowing calling classes to be referenced in static contexts rather than defining classes. 1) The parsing process is performed at runtime, 2) Look up the call class in the inheritance relationship, 3) It may bring performance overhead.

Sending JSON data using PHP's cURL library In PHP development, it is often necessary to interact with external APIs. One of the common ways is to use cURL library to send POST�...
