是否有一种简单的方法来访问包含ul和li标签的另一个div内的div标签?

时间:2019-07-08 07:29:57

标签: python html web-scraping tags

我正在尝试使用python和漂亮的汤从网站上刮取商店清单(该清单只能在您输入邮政编码时显示)。当我查看页面源代码或期望元素具有这种结构时

<div id ="storelist" class>
   <ul> 
        <li id ="00021455" class>
         .
         .
         .
              <div class ="wr-store-details">
                  <h3> class "Store abc"</h3>
                  <span> class ="20005 dc">
                  <span> class ="LA">
    </ul>
</div>

我正在努力访问第二个<div class ="wr-store-details">来抓取他的3个元素。

我尝试运行以下脚本,但实际上

from urllib.request import urlopen as uReq
from bs4 import BeautifulSoup as soup
import csv 
import requests 

URL = "http:www.abc.com"
r = requests.get(URL) 

soup = BeautifulSoup(r.content, 'html5lib') 

store_details_tab = []  # a list to store store_details-tab 

table = soup.find('div', attrs = {'id':'storelist'}) 

for row in table.findAll('div', attrs = {'class':'store_details'}): 
    store_details = {} 
    store_details['store_name'] = row.div.text 
    store_details['store_address'] = row.span.text 
    store_details['store_city'] = row.span.text 
    store_details_tab.append(store_details) 

filename = 'store_details_tab.csv'
with open(filename, 'wb') as f: 
    w = csv.DictWriter(f,['store_name','store_address','store_city']) 
    w.writeheader() 
    for store_details in store_details_tab: 
        w.writerow(store_details)

我希望使用具有store_name,store_city和store_zip的csv文件

  

错误:TypeError:需要一个类似字节的对象,而不是'str'


0 个答案:

没有答案