Question

我正在尝试抓取网站并将数据写入CSV。问题在于，由于我使用的是嵌套循环，因此所有数据都不会以csv结尾。

import requests 
from bs4 import BeautifulSoup
from csv import writer

with open ("full_links_details.csv", 'w', newline='') as csv_file: 
    csv_writer = writer(csv_file)
    csv_writer.writerow(["Details", "Details_Link", "image_link"])

    z = """
         <div class="container container1">
           <ul class="splist-view">
               <li class="wow fadeInUp">
                   <div class="row">
                       <div class="pic">
                           <a href="some_link_a">
                               <img src="some_image_link_a">
                           </a>
                       </div>
                       <div class="detail">
                           <ul>
                               <li class="hd"><a href="some_link_a">SomeTitleText-A</a></li>
                           </ul>
                       </div>
                   </div>
               </li>

               <li class="wow fadeInUp">
                   <div class="row">
                       <div class="pic">
                           <a href="some_link_b">
                               <img src="some_image_link_b">
                           </a>
                       </div>
                       <div class="detail">
                           <ul>
                               <li class="hd"><a href="some_link_b">SomeTitleText-B</a></li>
                           </ul>
                       </div>
                   </div>
               </li>
           </ul>
          </div>
        """
    souped_html_data = BeautifulSoup(z, "html.parser")

    div_detail_list = souped_html_data.find_all("div", "detail")
    div_pic_list = souped_html_data.find_all("div", "pic")

    for div_detail in div_detail_list:
        details = div_detail.get_text()


    for div_link in div_detail_list:
        div_link_a = div_link.find_all('a')
        for div_link_href in div_link_a:
            div_link_href_url = div_link_href.get('href')

    for div_pic in div_pic_list:
        div_pic_a = div_pic.find_all('img')
        for div_pic_a_src in div_pic_a:
            div_pic_a_src_link = div_pic_a_src.get('src')

        csv_writer.writerow([details, div_link_href_url, div_pic_a_src_link])

现在，无论我做什么，我都无法以正确的形式获取所有数据。我的意思是，如果我更改最后一行的缩进---有时重复detail变量，有时则重复前两个字段。因此，我确定嵌套的For Loop会造成问题。有什么办法可以使所有循环都达到同一级别，然后再写入数据？我认为，这将解决问题。

在@Joël的帮助下解决。将所有循环合并为：

    for div_detail, div_link, div_pic in zip(div_detail_list, div_detail_list, div_pic_list):
        details = div_detail.get_text()

        div_link_a = div_link.find_all('a')
        for div_link_href in div_link_a:
            div_link_href_url = div_link_href.get('href')

        div_pic_a = div_pic.find_all('img')
        for div_pic_a_src in div_pic_a:
            div_pic_a_src_link = div_pic_a_src.get('src')

        csv_writer.writerow([details, div_link_href_url, div_pic_a_src_link])

Answer 1

为了并行使用多个循环中的项目，可以使用内置的zip函数：

>>> seq0 = [1, 2, 3]
>>> seq1 = ['a', 'b', 'c']
>>> for s0, s1 in zip(seq0, seq1):
...    print(f"{s0} - {s1}") 
1 - a
2 - b
3 - c

但是，我觉得有些奇怪：您在三个不同的项目集上使用了三个循环，但是您确定其中三个循环的长度相同吗？

Python：使用来自多个嵌套循环的数据编写csv

1 个答案: