我尝试将两个Web抓取列表合并到一个列表中,但它只是显示一个实例。 (我已经有了名单和地址列表,我只想加入它们。)
/fonts/vendor/ionicons/dist/
第一份清单:
from bs4 import BeautifulSoup
import urllib.request
def get_HTML(url):
response = urllib.request.urlopen(url)
html = response.read()
return html
输出:
venues_html = get_HTML('http://www.cxra.com/venues/new-york/')
soup = BeautifulSoup(venues_html, "lxml")
for venue in soup('a', attrs={'href' : '#', 'onclick' : 'return false;'}):
display (venue.text)
第二个清单
'Manhattan Center Studios'
'Ellis Island'
'The TimesCenter'
'The Altman Building'
'NYIT Auditorium on Broadway'
输出:
for info in soup.findAll('div', attrs={'class' : 'infoUnit col-md-6'}):
display (info.text)
尝试加入两者:
'\n \n311 West 34th Street\r\nNew York City, 94710\n\n212.613.5536\n'
'\n \nEllis Island\r\nNew York, NY 10004\n\n212.613.5535\n'
'\n \n242 W 41st St\r\nNew York, NY 10036\n\n212.613.5535\n'
'\n \n135 W 18th St\r\nNew York, NY 10011\n\n212.613.5535\n'
'\n \n1871 Broadway\r\nNew York, NY 10023\n\n212.613.5536\n'
输出:
print ("Venue: " + venue.text + info.text)
我希望它能够为所有不同的场地做到这一点,而不仅仅是一个。我试过循环,但它们似乎只是反复显示一个实例。
答案 0 :(得分:0)
如果不确切知道display()的作用,我的猜测是你没有打印所有"实例"因为你只是从两个for循环的最后一次迭代中得到结果。
如果没有结构变化,最快的方法是将两个bs4剪贴簿的输出保存到两个列表中,并使用循环显示其内容。
from bs4 import BeautifulSoup
import urllib.request
import sys
def get_HTML(url):
response = urllib.request.urlopen(url)
html = response.read()
return html
venues_html = get_HTML('http://www.cxra.com/venues/new-york/')
soup = BeautifulSoup(venues_html, "lxml")
print('data scrapped')
list1 = []
list2 = []
for venue in soup('a', attrs={'href' : '#', 'onclick' : 'return false;'}):
list1.append(venue.text)
for info in soup.findAll('div', attrs={'class' : 'infoUnit col-md-6'}):
list2.append(info.text)
if len(list1) != len(list2):
print('Two lists not aligned, aborting')
sys.exit(0)
for index in range(len(list1)):
try:
print ("Venue: ", list1[index], list2[index])
except Exception as e:
print(e)