How to use curl to simulate login segmentfault in PHP?

<code>$ch = curl_init();
$url = "https://segmentfault.com/api/user/login?_=fb8eb567a6f2a4ed7d28a1ac62c0d018";

$data = array(
    'mail' => ***
    'password' => ***
);
foreach ($data as $key => $value){
    $postfields .= urlencode($key) . '=' . urlencode($value) . '&';
}
$postfields = rtrim($postfields, '&');
$headers = array(
    'Accept:*/*',
    'Accept-Encoding:gzip, deflate',
    'Accept-Language:zh-CN,zh;q=0.8',
    'Connection:keep-alive',
    'Content-Length:49',
    'Content-Type:application/x-www-form-urlencoded; charset=UTF-8',
 'Cookie:mp_18fe57584af9659dea732cf41c1c0416_mixpanel=%7B%22distinct_id%22%3A%20%22153c6c3ec0c91-04fd9c038-12771e2d-1fa400-153c6c3ec0d18a%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; PHPSESSID=web2~dom8lkdgosec57oljs98g2m8k0; _gat=1; Hm_lvt_e23800c454aa573c0ccb16b52665ac26=1463986883,1464937399,1465290769,1465713371; Hm_lpvt_e23800c454aa573c0ccb16b52665ac26=1465717067; _ga=GA1.2.1469164019.1455850659',
    'Host:segmentfault.com',
    'Origin:https://segmentfault.com',
    'Referer:https://segmentfault.com/',
    'User-Agent:Mozilla/5.0 (X11; Linux i686 (x86_64)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36',
    'X-Requested-With:XMLHttpRequest',
);

curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
curl_setopt($ch, CURLOPT_ENCODING, "");
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);


返回结果说我用户或密码错误？ 账号密码是正确的。</code>

Reply content:

<code>$ch = curl_init();
$url = "https://segmentfault.com/api/user/login?_=fb8eb567a6f2a4ed7d28a1ac62c0d018";

$data = array(
    'mail' => ***
    'password' => ***
);
foreach ($data as $key => $value){
    $postfields .= urlencode($key) . '=' . urlencode($value) . '&';
}
$postfields = rtrim($postfields, '&');
$headers = array(
    'Accept:*/*',
    'Accept-Encoding:gzip, deflate',
    'Accept-Language:zh-CN,zh;q=0.8',
    'Connection:keep-alive',
    'Content-Length:49',
    'Content-Type:application/x-www-form-urlencoded; charset=UTF-8',
 'Cookie:mp_18fe57584af9659dea732cf41c1c0416_mixpanel=%7B%22distinct_id%22%3A%20%22153c6c3ec0c91-04fd9c038-12771e2d-1fa400-153c6c3ec0d18a%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; PHPSESSID=web2~dom8lkdgosec57oljs98g2m8k0; _gat=1; Hm_lvt_e23800c454aa573c0ccb16b52665ac26=1463986883,1464937399,1465290769,1465713371; Hm_lpvt_e23800c454aa573c0ccb16b52665ac26=1465717067; _ga=GA1.2.1469164019.1455850659',
    'Host:segmentfault.com',
    'Origin:https://segmentfault.com',
    'Referer:https://segmentfault.com/',
    'User-Agent:Mozilla/5.0 (X11; Linux i686 (x86_64)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36',
    'X-Requested-With:XMLHttpRequest',
);

curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
curl_setopt($ch, CURLOPT_ENCODING, "");
$result = curl_exec($ch);
curl_close($ch);
var_dump($result);


返回结果说我用户或密码错误？ 账号密码是正确的。</code>

Foreword

This article comes from my answer on segmentfault, and I recorded the exciting parts in this blog.
The general meaning is to simulate logging in to segmentfault.com, and my hands are itchy for a while. This article will guide everyone to realize this operation.

Analysis

This is a very good question, but unfortunately, everyone’s replies are all just talk on paper without discussion. The answer with the most votes at the top actually said to give up the packet capture tool, which is simply ridiculous. You can see it directly by running F12 in chrome. The account and password are sent in clear text, why do we need to capture the packet? The http header of the other subject is copied from chrome.
According to my judgment, the cause of your problem is sending too many http headers, of which Content-Length is obviously problematic. This represents the content length. The packet you captured this time was 49, but next time you need to change your account and password. But that’s not necessarily the case. For example, if the account password is too long, it may result in truncation, and an incorrect password will be prompted anyway (because only part of the password is sent).

Plan

In fact, in order to explore this interesting question, I conducted an interesting experiment.
Here we use the node.js model in the simplest scripting language ajax to reconstruct the operation process.

Analysis

Let’s first go to the login page - the source page to take a rough look, among which

<script crossorigin src="https://dfnjy7g2qaazm.cloudfront.net/v-575e20ec/user/script/login.min.js"></script>

This cross-domain request loads a js script. Judging from the name, it should be related to login. We use it here to try to access it. The result is a mess.
According to the naming convention, we guessed that the name before compression might have been called login.js. Let’s see if it has been deleted. We tried to access https://dfnjy7g2qaazm.cloudfront.net/v-575e20ec/user/script/login.js, and it’s still there. It seems that their publisher may not be a Virgo. of.
Well let’s look down here:

$("form[action='/api/user/login']").submit(function() {
  var data, url;
  url = '/api/user/login';
  data = $(this).serialize();
  $.post(url, data, function(d) {
    if (!d.status) {
      return location.href = d.data;
    }
  });
  return false;
});

The code is very simple. We know that when the status in the request result is 0, it means the login is successful. At the same time, we also know that the background execution login request page is /api/user/login, that is, https://segmentfault.com/api/user/login. Let’s visit it, and it will get 404. This shows that the server has verified the input and determined that our request does not comply with normal logic. Next we start to forge request headers.

Request header

We use a modern browser similar to chrome to access https://segmentfault.com/user/login normally, press F12, select the network panel to start monitoring requests, then we fill in the account and password at will, and click to log in.
At this time there will be a message below, we extract the Request Header as follows

POST /api/user/login?_=93e1b923149fb56c4fd329fe95ea4001 HTTP/1.1
Host: segmentfault.com
Connection: keep-alive
Content-Length: 46
Pragma: no-cache
Cache-Control: no-cache
Accept: */*
Origin: https://segmentfault.com
X-Requested-With: XMLHttpRequest
User-Agent: xxxx
Content-Type: application/x-www-form-urlencoded; charset=UTF-8
DNT: 1
Referer: https://segmentfault.com/
Accept-Encoding: gzip, deflate, br
Accept-Language: en-US,en;q=0.8,zh-CN;q=0.6,zh;q=0.4
Cookie: PHPSESSID=web5~to8l5ovmt9t3jkb84aevuqf151; Hm_lvt_e23800c454aa573c0ccb16b52665ac26=1465799317; Hm_lpvt_e23800c454aa573c0ccb16b52665ac26=1465799317; _ga=GA1.2.915515414.1465799317; _gat=1

We only need to send these requests to the server in the same way. In theory, there will be no problems, and there will be no more 404s.
Among the data here, some do not need to be sent, and some must be sent.
We can test them one by one.

Debugging

We use nodejs here to simply write a piece of code to test the parameters verified by the server.
The boring test is to keep deleting requests to see if the server will return 404.
The process will not be described in detail, the result is:

1. The _ in querystring must correspond to the Cookie in PHPSESSID.

The value of

2. X-Requested-With needs to have the ajax request flag, that is, XMLHttpRequest

The value of

3. Referer

It seems that their server is quite strict.

Source code

var superagent = require('superagent');


superagent.post('https://segmentfault.com/api/user/login?_=7ef046ad4f224034d7b51655238bd870')
    .set('Referer', 'https://segmentfault.com/user/login')
    .set('X-Requested-With', 'XMLHttpRequest')
    .set('Cookie', 'PHPSESSID=web1~395mahoqliohh5kclv894ibpr3; _gat=1; _ga=GA1.2.1234754628.1465797373; Hm_lvt_e23800c454aa573c0ccb16b52665ac26=1465797373; Hm_lpvt_e23800c454aa573c0ccb16b52665ac26=1465797538')
    .send({
        mail: "xxxxxx",
        password: "xxxx"
    })
    .type('form')
    .end(function(err, res) {
        if (err || !res.ok) {
            console.log(err.status);
        } else {
            console.log('yay got ' + JSON.stringify(res.body));
        }
    });

At the same time, the open source is on github at the address segmentfault_loginer

The first part is not encrypted, 90% of the possibility is caused by cookies

Give you a suggestion:

1. Now visit: https://segmentfault.com/ and get the cookie
2. Then submit the post
3._This GET parameter is also very important

You can use Charles to capture HTTPS packets. You can refer to this http://www.tuicool.com/articles/JfEZr23
If the password is encrypted with JS on the front end, you can find this encryption code. You can also find out the encryption algorithm and you can simulate this algorithm for encryption. I also encountered this problem before when I crawled the China Mobile website

看原始http请求

<code>$ch = curl_init();
$url = "https://segmentfault.com/api/user/login?_=259f90fcf626f304c69c52db1454f03e";

$data = array(
    'mail' => '***',
    'password' => '**',
);
foreach ($data as $key => $value){
    $postfields .= urlencode($key) . '=' . urlencode($value) . '&';
}
$postfields = rtrim($postfields, '&');
$headers = array(
    'Accept:*/*',
    'Accept-Encoding:gzip, deflate',
    'Accept-Language:zh-CN,zh;q=0.8',
    'Connection:keep-alive',
    'Content-Type:application/x-www-form-urlencoded; charset=UTF-8',
    'Cookie:mp_18fe57584af9659dea732cf41c1c0416_mixpanel=%7B%22distinct_id%22%3A%20%22153c6c3ec0c91-04fd9c038-12771e2d-1fa400-153c6c3ec0d18a%22%2C%22%24initial_referrer%22%3A%20%22%24direct%22%2C%22%24initial_referring_domain%22%3A%20%22%24direct%22%7D; editor-code-detect-disabled=1; PHPSESSID=web2~oag2uol7e47i88hp6t6uqac9b0; Hm_lvt_e23800c454aa573c0ccb16b52665ac26=1465290769,1465713371,1465781816,1465866651; Hm_lpvt_e23800c454aa573c0ccb16b52665ac26=1465867161; _ga=GA1.2.1469164019.1455850659; _gat=1',
    'Host:segmentfault.com',
    'Origin:https://segmentfault.com',
    'Referer:https://segmentfault.com/',
    'User-Agent:Mozilla/5.0 (X11; Linux i686 (x86_64)) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/48.0.2564.97 Safari/537.36',
    'X-Requested-With:XMLHttpRequest',
);

curl_setopt($ch, CURLOPT_URL, $url);
curl_setopt($ch, CURLOPT_POST, 1);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_HTTPHEADER, $headers);
curl_setopt($ch, CURLOPT_POSTFIELDS, $postfields);
//curl_setopt($ch, CURLOPT_SSL_VERIFYPEER, FALSE);
curl_setopt($ch, CURLOPT_ENCODING, "");
$result = curl_exec($ch);
curl_close($ch);

var_dump($result);</code>

至少得先保存个ｃｏｏｋｉｅ在登录吧

之前写过一个51job的模拟登录并投递简历的demo，代码如下，可参考

<?php
// 首先引入zend框架（官网地址：http://www.zend.com/）
require_once 'Zend/Loader.php';

Zend_Loader::loadClass('Zend_Http_Client');
$client  = new Zend_Http_Client();
$client->setCookieJar();

/**
 * 登录
 */
//$client->setHeaders('Set-cookie', $response->getHeader('Set-cookie'));
$client->setUri('http://my.51job.com/my/passport_login.php');
$client->setMethod('POST');
$client->setParameterPost(array(
    'from_domain'        => 'www.51job.com',
    'passport_loginName' => '*****',  // 账号
    'passport_password'  => '***'   // 密码
));
$res = $client->request();

/**
 * 投简历
 */
$client->setUri('http://my.51job.com/sc/sendtwo/send_resume_new.php');
$client->setHeaders('Host', 'my.51job.com');
$client->setHeaders('Cookie', $res->getHeader('Set-cookie'));
$client->setMethod('GET');
$client->setParameterGet(array(
    'isEN'         => '0',
    'rsmid'        => '337097130',
    'deflang'      => '0',
    'coverid'      => '',
    'jsoncallback' => 'jsonp1447931687767',
    '_'            => '1447931692678',
    'jobiduni'     => '(73015544)',   //括号里面为职位id
));
$rs = $client->request();
print_r($rs);

不错，问题具有代表性

既然是https，那为啥不加上curl的https选项

应该是密码在前端做了hash加密，你可以抓个包看下