Question

import requests
from bs4 import BeautifulSoup
import csv

url='https://stackoverflow.com/users'
response= requests.get(url)
html=response.content

soup= BeautifulSoup(html, 'html.parser')

divs= soup.find_all("div", class_='container')

with open('stackdata.csv', 'a') as csv_file:
    writer=csv.writer(csv_file)
    for div in divs:
        print(div.text)
        name_box=soup.find('div', attrs={'class': 'user-details'})
        name=name_box.text
        print(name)
        writer.writerow([name])

当我使用div.text时，它会打印所有数据，但是当我想在csv文件中写入这些数据时，它只会传输一个上层数据。

Answer 1

我认为这就是你想要刮刀行为的方式。运行它并找到填写了数据的csv文件。

import requests
from bs4 import BeautifulSoup
import csv

response = requests.get('https://stackoverflow.com/users').text
soup = BeautifulSoup(response, 'lxml')
for items in soup.select('.user-details'):
    name = items.select("a")[0].text
    location = items.select(".user-location")[0].text
    reputation = items.select(".reputation-score")[0].text
    print(name,location,reputation)

    with open('stackdata.csv','a',newline='') as csv_file:
        writer = csv.writer(csv_file)
        writer.writerow([name,location,reputation])

刮掉Stack Overflow用户数据

1 个答案: