网络抓取工具Python AttributeError:“ NoneType”对象没有属性“获取”

时间:2020-05-25 02:44:52

标签: web-scraping beautifulsoup python-3.7 zillow

嗨,谁能解决这个问题。

url =“ https://www.zillow.com/walnut-ca/?searchQueryState=%7B%22pagination%22%3A%7B%7D%2C%22usersSearchTerm%22%3A%22Walnut%2C%20CA%22%2C%22mapBounds%22%3A%7B%22west%22%3A-117.93482729053105%2C%22east%22%3A-117.75286623096073%2C%22south%22%3A33.93783156520187%2C%22north%22%3A34.06392018234896%7D%2C%22isMapVisible%22%3Atrue%2C%22mapZoom%22%3A12%2C%22filterState%22%3A%7B%22price%22%3A%7B%22min%22%3A400000%2C%22max%22%3A700000%7D%2C%22mp%22%3A%7B%22min%22%3A1448%2C%22max%22%3A2535%7D%2C%22sort%22%3A%7B%22value%22%3A%22globalrelevanceex%22%7D%7D%2C%22isListVisible%22%3Atrue%7D

d = {'键':'值'} 打印(d) d ['new key'] ='新值' 打印(d)

query_houses = {} house_no = 0 而True:

response = requests.get(url)
data = response.text
soup = BeautifulSoup(data,'html.parser')
houses = soup.find_all('article',{'class':'list-card list-card-short list-card_not-saved'})

for house in houses:
    location = house.find('address',{'class': 'list-card-addr'})
    value = house.find('div',{'class': 'list-card-price'})
    detail = house.find('ul', {'class':'list-card-details'})
    seller = house.find('div',{'class':'list-card-truncate'})
    link = house.find('a', {'class': 'list-card-link'}).get('href')

    house_response = requests.get(link)
    house_data = house_response.text
    house_soup = BeautifulSoup(house_data, 'html.parser')
    square = house_soup.find('span',{'class':'ds-bed-bath-living-area'})
    year_build = house_soup.find('span',{'class':'ds-body ds-home-fact-value'})
    estimated_sales_range = house_soup.find('div',{'class':'Spacer-sc-17suqs2-0 pfWXf'})

    house_no+=1
    query_houses[house_no] = [location, value, detail, seller, link, square, year_build, estimated_sales_range]

url_tag = soup.find('a',{'title':'Next-page'})
if url_tag.get('href'):
    url= 'https://zillow.com' + url_tag.get('href')
    print(url)
else:
    break     

0 个答案:

没有答案