Question

我希望我的程序将文章日期，标题和正文文本写入csv文件。当我在控制台中打印正文时，它会打印所有内容，但是在csv文件中，它只会打印文章的最后一行。

CSV结果：

控制台打印：

我尝试将日期，标题和正文文本写成一行，并分别放置到列表中，结果具有相同的结果。

from bs4 import BeautifulSoup
from urllib.request import urlopen
import csv

csvfile = "C:/Users/katew/Dropbox/granularitygrowth/Politico/pol.csv"
with open(csvfile, mode='w', newline='') as pol:
    csvwriter = csv.writer(pol, delimiter='|', quoting=csv.QUOTE_MINIMAL)
    csvwriter.writerow(["Date", "Title", "Article"])

    #for each page on Politico archive
    for p in range(0,1):
        url = urlopen("https://www.politico.com/newsletters/playbook/archive/%d" % p)
        content = url.read()

        #Parse article links from page
        soup = BeautifulSoup(content,"lxml")
        articleLinks = soup.findAll('article', attrs={'class':'story-frag format-l'})

        #Each article link on page
        for article in articleLinks:
            link = article.find('a', attrs={'target':'_top'}).get('href')

            #Open and read each article link
            articleURL = urlopen(link)
            articleContent = articleURL.read()

            #Parse body text from article page
            soupArticle = BeautifulSoup(articleContent, "lxml")

            #Limits to div class = story-text tag (where article text is)
            articleText = soupArticle.findAll('div', attrs={'class':'story-text'})
            for div in articleText:

                #Find date
                footer = div.find('footer', attrs={'class':'meta'})
                date = footer.find('time').get('datetime')
                print(date)

                #Find title
                headerSection = div.find('header')
                title = headerSection.find('h1').text
                print(title)

                bodyText = div.findAll('p')
                for p in bodyText:
                    p_string = str(p.text)
                    textContent = "" + p_string
                    print(textContent)

                #Adds data to csv file
                csvwriter.writerow([date, title, textContent])

我希望csv文件包含日期，标题和全文。

Answer 1

问题出在您的for p in bodyText:循环中。您正在将最后一个p的文本分配给您的textContent变量。尝试类似的东西：

textContent = ""
bodyText = div.findAll('p')
for p in bodyText:
    p_string = str(p.text)
    textContent += p_string + ' '

print(textContent)
csvwriter.writerow([date, title, textContent])

如何解决写入csv文件的问题

1 个答案: