Question

我想要这样的数据...

“基本球衣

听话了吗

主要：100％棉。”

在一个单元格中，但是我正在获取这样的数据...

“基本球衣在锡罐上的字样是：100％棉。”

这就是HTML

<div class="about-me">
    <h4>ABOUT ME</h4>
    <span><div>Basic jersey</div><div>Does what it says on the tin</div><br>Main: 100% Cotton.</span>
</div>

这是我的密码

from selenium import webdriver
from lxml import html
import pandas as pd
import collections, os
from bs4 import BeautifulSoup

def Save_to_Csv(data):
    filename = 'data.csv'
    df = pd.DataFrame(data)
    df.set_index('Title', drop=True, inplace=True)
    if os.path.isfile(filename):
       with open(filename,'a') as f:
           df.to_csv(f, mode='a', sep=",", header=False, encoding='utf-8')
    else:
        df.to_csv(filename, sep=",", encoding='utf-8')

with open('urls.txt', 'r') as f:
        links = [link.strip() for link in f.readlines()]
driver = webdriver.Chrome()
for urls in links:
    global image
    driver.get(urls)
    source = driver.page_source
    tree = html.fromstring(source)
    data = BeautifulSoup(source, 'html.parser')
    imgtag = data.find_all('li', attrs={'class':'image-thumbnail'})
    image = []
    for imgsrc in imgtag:
        image.append(imgsrc.img['src'].replace('?$S$&wid=40&fit=constrain', '?$XXL$&wid=513&fit=constrain'))
    title = tree.xpath('string(.//div/h1)')         
    price = tree.xpath('string(.//span[@class="current-price"])')
    sku = tree.xpath('string(.//div[@class="product-code"]/span)')
    aboutme = tree.xpath(('string(.//div[@class="about-me"]/span)'))

    foundings = collections.OrderedDict()
    foundings['Title'] = [title]
    foundings['Price'] = [price]
    foundings['Product_Code'] = [sku]
    foundings['Abouy_Me'] = [aboutme]
    foundings['Image'] = [image]
    Save_to_Csv(foundings)

    print title, price, sku, aboutme, image
driver.close()

Answer 1

使用您提供的HTML，可以使用stripped_strings生成器来解决此问题，如下所示：

这将使每个组件都位于剥离列表中，然后将它们与换行符连接在一起

from bs4 import BeautifulSoup

html = """
<div class="about-me">
    <h4>ABOUT ME</h4>
    <span><div>Basic jersey</div><div>Does what it says on the tin</div><br>Main: 100% Cotton.</span>
</div>"""

soup = BeautifulSoup(html, "html.parser")

print('\n'.join(soup.span.stripped_strings))

将数据另存为新行，但保存在单个单元格lxml python

1 个答案: