使用特定列将抓取的数据导出到CSV

时间:2019-04-04 15:29:34

标签: python beautifulsoup export-to-csv

我的代码当前正在命令屏幕上打印结果。

所需的结果(请参见所附的屏幕截图):将最终输出写入“ a2”列中的CSV文件 并将sku#输出到列“ a1” sku#始终是网址中第5个“ /”之后的文本

这是代码

from bs4 import BeautifulSoup
import urllib.request
import csv
def get_bullets(url):
    page = urllib.request.urlopen(url)
    soup = BeautifulSoup(page,'lxml')
    content = soup.find('div', class_='js-productHighlights product-highlights c28 fs14 js-close')
    bullets = content.find_all('li', class_='top-section-list-item')
    for bullet in bullets:
     print(bullet.string)

get_bullets('https://www.bhphotovideo.com/c/product/1225875-REG/canon_1263c004_eos_80d_dslr_camera.html')

所需结果:

enter image description here

谢谢!

1 个答案:

答案 0 :(得分:1)

from bs4 import BeautifulSoup
import urllib.request
import pandas as pd


def get_bullets(url):
    sku = url.split('/')[5]
    page = urllib.request.urlopen(url)
    soup = BeautifulSoup(page,'lxml')
    content = soup.find('div', class_='js-productHighlights product-highlights c28 fs14 js-close')
    bullets = content.find_all('li', class_='top-section-list-item')

    bullets_text = '\n'.join([ bullet.text for bullet in bullets ])

    temp_df = pd.DataFrame([[sku, bullets_text]], columns = ['sku','bullets'])
    temp_df.to_csv('path/filename.csv', index=False)


get_bullets('https://www.bhphotovideo.com/c/product/1225875-REG/canon_1263c004_eos_80d_dslr_camera.html')