Question

我正在试图刮取汽车数据。他们的＆＃39; id＆＃39;标签增加1，但我似乎无法弄清楚如何做到这一点。这就是我所拥有的：

import bs4 as bs
import urllib

source = urllib.request.urlopen('http://www.25thstauto.com/inventory.aspx?cursort=asc&pagesize=500').read()
soup = bs.BeautifulSoup(source, 'lxml')

#finds the total number of cars
count = soup.find('span', {'id': 'ctl00_cphBody_inv1_lblVehicleCount'}).getText()[:2]
count = int(count)

i = 1
for url in range(1,count):
url = soup.find_all('a', {'id': 'ctl00_cphBody_inv1_rptInventoryNew_ctl0'+i+'_nlVehicleDetailsTitle'})
    print(url['href'])
    i = i + 1

Answer 1

import bs4 as bs
import urllib
import re

source = urllib.request.urlopen('http://www.25thstauto.com/inventory.aspx?cursort=asc&pagesize=500').read()
soup = bs.BeautifulSoup(source, 'lxml')

for a in soup.find_all('a', id=re.compile('ctl00_cphBody_inv1_rptInventoryNew')):
    print(a.get('href'))

出：

2008_Chevrolet_Malibu_Easton_PA_265928462.veh
2008_Chevrolet_Malibu_Easton_PA_265928462.veh
2008_Chevrolet_Malibu_Easton_PA_265928462.veh
2002_Nissan_Xterra_Easton_PA_266894015.veh
2002_Nissan_Xterra_Easton_PA_266894015.veh
2002_Nissan_Xterra_Easton_PA_266894015.veh
2009_Chevrolet_Cobalt_Easton_PA_265621796.veh
2009_Chevrolet_Cobalt_Easton_PA_265621796.veh

使用regex查找a属性包含id

的ctl00_cphBody_inv1_rptInventoryNew代码

或使用CSS selector：

for a in soup.select('a[id*=ctl00_cphBody_inv1_rptInventoryNew]'):
    print(a.get('href'))

这个想法是一样的。

简单的Python Web-scraper与美丽的汤

1 个答案: