我想抓数据,但我只能第一次成功请求,当我重新跑步时,我再也无法获得身体。
我的代码:
url="http://www.carparts.com/results/?N=0&Nr=AND%28universal%3A0%29&Ntk=Main&Ntx=mode+matchallany&Nty=1&PN=0+5727&VN=4294953018+4294962799+4294962221+4294957507+4294965468&universal=0"
request_headers = {
"Accept-Language": "en-US,en;q=0.5",
"User-Agent": "Mozilla/5.0 (Windows NT 10.0; WOW64; rv:40.0) Gecko/20100101 Firefox/40.0",
"Accept": "text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8",
"Cache-Control": "no-cache, no-store, must-revalidate,post-check=0, pre-check=0",
"Content-Length":"22035",
"keep_alive":"false",
"Content-Type":"text/html; charset=UTF-8",
"Vary":"Accept-Encoding",
"Pragma":"no-cache"
}
pageSkimData = requests.get(url, headers=request_headers)
treeSkimData = html.fromstring(pageSkimData.content)
答案 0 :(得分:0)
它也发生在我身上,一旦我做了一个初始请求,源代码稍微改变了所有后续请求,获取你可以忘记结果的数据并使用产品 divs:
pageSkimData = requests.get(url, headers=request_headers)
treeSkimData = html.fromstring(pageSkimData.content)
print(pageSkimData.content)
for prod in treeSkimData.xpath("//div[@class='product icon-chevron-small-right prodwithad clearfix']"):
print(prod.xpath("./div[@class='producttitle']/h2/a/text()")[0])
for li in prod.xpath("//ul[@class='product-feature-list']/li/span"):
print(li.xpath("./strong/text()")[0] + li.xpath("./span/text()")[0])
输出:
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Blue, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Purple, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Yellow, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Blue, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Blue, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Purple, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Blue, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Blue, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock
Plasmaglow Light Bulb - Pink, Direct Fit
Color/ Finish:White
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-WH
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-BL
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H3
Part Number:H3P
Availability: In Stock
Color/ Finish:Yellow
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-YW
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H7
Part Number:H7L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9006
Part Number:9006H
Availability: In Stock
Color/ Finish:Purple
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PR
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:9007
Part Number:9007L
Availability: In Stock
Color/ Finish:Blue
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:H1
Part Number:H1L
Availability: In Stock
Color/ Finish:Pink
Product Fit:Direct Fit
Quantity Sold:Set of 2
Anticipated Ship Out Time:1-2 Business Days
Bulb Type:194/168
Part Number:194LED-PI
Availability: In Stock