Question

我试图使用dryscrape和python来学习http://quotes.toscrape.com/用于学习目的。我能够通过class =＆＃34;引用＆＃34;获得所有div。想用class =＆＃34; quote＆＃34;循环遍历div列表。并使用xpath从此父元素获取多个数据。

import dryscrape
from bs4 import BeautifulSoup
session = dryscrape.Session()
url = 'http://quotes.toscrape.com/'
print 'Visiting the URL...'
session.visit(url)
print 'Status: ', session.status_code()
for div in session.xpath("//div[@class='quote']"):
    # please help me to scrape author and quote for each div elements

Answer 1

import requests
from bs4 import BeautifulSoup
url = 'http://quotes.toscrape.com/'
r = requests.get(url)
soup = BeautifulSoup(r.text)
for div in soup.findAll("div", {"class": "quote"}):
  print('Quote : ' + div.find('span').get_text())
  print('Author : ' + div.find('small').get_text())

Answer 2

我们可以遍历每个xpath元素，这些元素将是具有单个元素内容的对象。每个对象都有获取数据的方法。

import dryscrape
session = dryscrape.Session()
url = 'http://quotes.toscrape.com/'
print 'Visiting the URL...'
session.visit(url)
print 'Status: ', session.status_code()
for div in session.xpath("//div[@class='quote']"):
    print "Quote: ", div.at_xpath(".//span").text()
    print "Author: ", div.at_xpath(".//small").text()

Dryscrape：使用xpath从父节点列表中抓取子节点数据

2 个答案: