Question

我正在尝试从aliexpress获取产品列表，但它无法将其重定向到主页我从中获取产品的网址列表如下

url="https://m.aliexpress.com/search.htm?keywords=mobile";

这是我的卷曲代码无法正常工作。

<?php

include('simple_html_dom.php');

$url = 'https://m.aliexpress.com/search.htm?keywords=mobile/';

$ch = curl_init();
curl_setopt($ch, CURLOPT_HEADER, 0);
curl_setopt($ch, CURLOPT_VERBOSE, 0);
curl_setopt($ch, CURLOPT_RETURNTRANSFER, true);
curl_setopt($ch, CURLOPT_URL, $url );
curl_setopt($ch, CURLOPT_FOLLOWLOCATION, true);
curl_setopt($ch, CURLOPT_CONNECTTIMEOUT, 200);


curl_setopt($ch, CURLOPT_TIMEOUT, 120);
curl_setopt($ch, CURLOPT_HTTPHEADER, array(
    ':authority:www.aliexpress.com',
    ':method:GET',
//  ':path:/item/2017-New-Summer-Fashion-Mens-T-Shirts-Slim-Fit-Short-Sleeve-T-Shirt-032/32798976881.html',
    ':scheme:https',
    'accept:text/html,application/xhtml+xml,application/xml;q=0.9,image/webp,*/*;q=0.8',
//  'accept-encoding:gzip, deflate, sdch, br',
    'accept-language:en-US,en;q=0.8',
    'cache-control:no-cache',
    'cookie:ali_apache_id=10.181.15.98.1496226301876.152542.8; ali_beacon_id=10.181.15.98.1496226301876.152542.8; aep_common_f=aEhfbgrZ7e4e9UmjuOWm1CANyaLLaqTDc80Oan6/v5duIrtz3XfvQQ==; cna=BIi1EWdCs1QCAS43pj2EYRUu; _uab_collina=149806153775875750712574; _mle_tmp0=eNrz4A12DQ729PeL9%2FV3cfUx8KvOTLFScnF1dDR1dDV2dLU0sDSxNLEwsTR0cjN2dHI0MTM0NzNW0kkusTI0sbQwMTczMjExtDDSSUxGE8itsDKojQIApLsX8Q%3D%3D; aep_history=keywords%5E%0Akeywords%09%0A%0Aproduct_selloffer%5E%0Aproduct_selloffer%0932788134848%0932798976881; xman_us_t=x_lid=bg1134819498lzch&sign=y&x_user=bUBd5Jk0irdh4KaPNADUTaZgZPbhimHjaVryhYI9YCI=&ctoken=18y_131g18r61&need_popup=y&l_source=aliexpress; xman_us_f=zero_order=n&x_locale=en_US&x_l=0&last_popup_time=1496226312427&x_user=BG|Nikola|Mihaylov|ifm|754321409&no_popup_today=n; intl_locale=en_US; aep_usuc_f=isfm=y&site=glo&c_tp=USD&x_alimid=754321409&isb=y&region=BG&b_locale=en_US; intl_common_forever=iP9SzYB6uR11n9PDuziWZV+m6wS5lewBhe0mchSYZzPnAbhr0LZabw==; _ga=GA1.2.1468541176.1496226329; _gid=GA1.2.15678206.1498472399; JSESSIONID=C8CB956B69CC08A9B351C25C869BBF29; _umdata=70724C19E34629158F306CE35FF4594FD67FF1CAB2345D48D8DBB0AA935079C88BF9397A2C0FD39DCD43AD3E795C914C85C011A26645999BFC392F47E8EAF27C; ali_apache_track=mt=1|ms=|mid=bg1134819498lzch; ali_apache_tracktmp=W_signed=Y; acs_usuc_t=acs_rt=4426ebd3d8e64e528a12a78ded645f48&x_csrf=o8z03s_owisa; xman_f=A2GX/Y1rs0cttixwSxevdzKgU2yRTOLeKBX7fdKnMDdXVLpqHphq8YkU+FAR6NBhi7hO5A7zc3/N6J/t/H9GWyHI+NwKJhVmLd4RElBNdA8pz3lnGKz0ftxea14UUDnFXkIy11hLAIx4XZqeotvVY5xqQ2mHDWxJdTYyFGUJp2NUu5sA+A+/LFaZo/EhWIfeloGoaQnpsd2ps2jdCJ6JyDiKEeiASt17bEr5yszxaPRfbd7hfZUA4PIxgI7v5xSjj851b0fxepl6UbWi17mb+nk2O1VDuqt3mITEb3d0ZfzzZpWy9G6CuVbj/20I8NLAlFY52SkZtltbyMAImZfaIaEFhz350zco; xman_t=jf4jAohRGS5aZgdMN923jjta0ojsSeWEwqlEWooob6QtKTtCgvI1+gCB8pCzof8e0Fa+5dsa4Qdi4uZStlIJ5bhyvUdv6xIqA03ua02HlZ8MwuJUAcQ1PvyeTvJiNRtUzKZ0KN4FeDGdPD02q9xuly6mAD7dPmkJ2xZpswuxBMKMJ7Cu0OGB//ooxRvcjMzbvPxf9q/Q/+uySj2V3tLsQ3uAMOETjIklOh/aoXkZNWDy9WX2fRsAanRANkHxb1MoL3hntz7XDtYCx1MZis9/cAsg76m72h+lz+6pSbIZeJlH6kt9G3JkNafhR9EOgaxiGS/dr3S1TVPKJ8Q58yZMKhjrNQ2yCNzpq1SkWZs7LMNIcGVvDq9kxhYryBgx9ADN4ezb4uT7kYAZaz09myq8tIdlP6P+QHHkADgkeSnsLmodjEednJop66zIkTrowbYq8s/5ggHXHqnfyvDmi/O/00thL/lNG05+KHaOU1VV1y7VXtxxDlWADzQwPk9/AnMWK3VV80qXOsSGXpycloxhz7agZlsrCRV6u+KhsaUqMd/yszE09+GNVwdN7/UeGAklUvUHUAIykqRkpdOp1z0Vi87QWFW5TOn850K36OgHVxdiYxldDmEewuXBMmrreBWz; __utmt=1; __utma=3375712.1468541176.1496226329.1498476312.1498478200.3; __utmb=3375712.5.10.1498478200; __utmc=3375712; __utmz=3375712.1498476312.2.2.utmcsr=google|utmccn=(organic)|utmcmd=organic|utmctr=(not%20provided); isg=AgcHaslcEs7cDpY8UNqaqtarlrsRpIzPoZDqYtndZhbiSDMK4NzOPvZIzj7t',
    'pragma:no-cache',
    'upgrade-insecure-requests:1',
    'user-agent:Mozilla/5.0 (Windows NT 6.1; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/58.0.3029.110 Safari/537.36',
));

$response = curl_exec($ch);
if($response == false){
    echo "Curl error: " . curl_error($ch) . "(" . curl_errno($ch) . ")";
    exit;
}
curl_close($ch);
$dom  = str_get_html($response);


echo $dom;

?>

Answer 1

这不是一个答案，但评论太大，无法放入评论栏

首先，您的HTTPHEADER没有任何意义，但这不是您的问题。问题是产品根本不在html中，而是通过javascript从https://m.aliexpress.com/search/mainsearch/asy/GetMainSearchDataJson.do获取

，但是为了从那里获取产品，你首先需要一个有效的会话cookie，以及一个名为pageId的令牌（不，它不是像数字那样明智的东西，它是某种哈希，最好的猜测，它可能是一个十六进制编码的SHA1哈希，无论出于何种原因），否则你会收到拒绝访问错误。问题是，这个pageId令牌是在浏览器中用javascript生成的。为了在PHP中计算pageId，您需要对计算pageId的javascript进行逆向工程，并在PHP中实现该算法。我认为生成pageId哈希的代码始于https://i.alicdn.com/aefe-mobile-global/standalone/??venus.e2980efc.js，但我不能为你进一步挖掘（除非你愿意为此付钱，ofc），祝你好运

Php cURL不能用于速卖通

1 个答案: