PHP uses curl proxy to implement data grabbing method

This article mainly introduces the method of using curl proxy to capture data in PHP, and analyzes the operation skills of PHP using curl proxy to capture data in the form of examples. Friends in need can refer to this article

The example describes how PHP uses curl proxy to capture data. Share it with everyone for your reference, the details are as follows:

<?php
define ( &#39;IS_PROXY&#39;, true ); //是否启用代理
function async_get_url($url_array, $wait_usec = 0)
{
  if (!is_array($url_array))
    return false;
  $wait_usec = intval($wait_usec);
  $data  = array();
  $handle = array();
  $running = 0;
  $mh = curl_multi_init(); // 开启多线程
  $i = 0;
  foreach($url_array as $url) {
    $ch = curl_init();
    if (IS_PROXY) {
    //以下代码设置代理服务器
    //代理服务器地址http://www.cnproxy.com/proxy1.html !!Hong Kong, China的速度比较好
    curl_setopt ($ch, CURLOPT_PROXY,&#39;110.4.12.170:80&#39; );
    }
    curl_setopt($ch, CURLOPT_URL, $url);
    curl_setopt($ch, CURLOPT_RETURNTRANSFER, 1); // return don&#39;t print
    curl_setopt($ch, CURLOPT_TIMEOUT, 30); //设置超时时间
    curl_setopt($ch, CURLOPT_USERAGENT, &#39;Mozilla/4.0 (compatible; MSIE 5.01; Windows NT 5.0)&#39;);
    curl_setopt($ch, CURLOPT_FOLLOWLOCATION, 1); // 302 redirect
    curl_setopt($ch, CURLOPT_MAXREDIRS, 7); //HTTp定向级别
    curl_multi_add_handle($mh, $ch); // 把 curl resource 放进 multi curl handler 里
    $handle[$i++] = $ch;
  }
  /* 执行 */
  do {
    $mrc = curl_multi_exec($mh, $running);
    if ($wait_usec > 0) /* 每个 connect 要间隔多久 */
      usleep($wait_usec); // 250000 = 0.25 sec
  } while ($mrc == CURLM_CALL_MULTI_PERFORM);
  while ($running && $mrc == CURLM_OK) {
    if (curl_multi_select($mh) != -1) {
      do {
        $mrc = curl_multi_exec($mh, $running);
      } while ($mrc == CURLM_CALL_MULTI_PERFORM);
    }
  }
  /* 读取资料 */
  foreach($handle as $i => $ch) {
    $content = curl_multi_getcontent($ch);
    $data[$i] = (curl_errno($ch) == 0) ? $content : false;
  }
  /* 移除 handle*/
  foreach($handle as $ch) {
    curl_multi_remove_handle($mh, $ch);
  }
  curl_multi_close($mh);
  return $data;
}
$urls = array(&#39;http://map.baidu.com&#39;);
$re = async_get_url($urls);
echo $re[0];
?>

The above is the entire content of this article, I hope it will be helpful to everyone’s study .

Related recommendations:

How to use curl to send requests in PHP

php implements the method of using curl to simulate ip and source for access

PHP implements curl or file_get_contents How to get the page that requires authorization

#

The above is the detailed content of PHP uses curl proxy to implement data grabbing method. For more information, please follow other related articles on the PHP Chinese website!