Question

我编写了以下代码，以使用BeautifulSoup-

从网站上获取产品说明。

def get_soup(url):
    try:
        response = requests.get(url)
        if response.status_code == 200:
            html = response.content
            return BeautifulSoup(html, "html.parser")
    except Exception as ex:
        print("error from " + url + ": " + str(ex))

def get_product_details(url):
    try:
        soup = get_soup(url)
        prod_details = dict()
        desc_list = soup.select('p ~ ul')
        prod_details['description'] = ''.join(desc_list)
        return prod_details
    except Exception as ex:
        logger.warning('%s - %s', ex, url)

if __name__ == '__main__':
    get_product_details("http://www.aprisin.com.sg/p-748-littletikespoptunesguitar.html")

在上面的代码中，我试图将description（一个列表）转换为字符串，但出现问题-

[WARNING] aprisin.py:82 get_product_details() : sequence item 0: expected str instance, Tag found - http://www.aprisin.com.sg/p-748-littletikespoptunesguitar.html

输出描述而不将描述转换为字符串-

[<ul>
<li>Freestyle</li>
<li>Play along with 5 pre-set tunes: </li>
</ul>, <ul>
<li>Each string will play a note</li>
<li>Guitar has a whammy bar</li>
<li>2-in-1 volume control and power button </li>
<li>Simple and easy to use </li>
<li>Helps develop music appreciation </li>
<li>Requires 3 "AA" alkaline batteries (included)</li>
</ul>]

Answer 1

您正在将tags（对象）列表而不是字符串传递给join()。 join()使用字符串列表。对联接功能使用以下代码更改：-

prod_details['description'] = ''.join([tag.get_text() for tag in desc_list])

或

prod_details['description'] = ''.join([tag.string for tag in desc_list])

如果您想要描述以及html内容，可以使用以下内容：-

# this will preserve the html tags and indentation.
prod_details['description'] = ''.join([tag.prettify() for tag in desc_list])

或

# this will return the html content as string.
prod_details['description'] = ''.join([str(tag) for tag in desc_list])

Answer 2

desc_list是bs4.element.Tag的列表。您应该将标签转换为字符串：

    desc_list = soup.select('p ~ ul')
    prod_details['description'] = str(desc_list[0])

Answer 3

您正在尝试加入标签列表，但是join方法需要str参数。试试：

''.join([str(i) for i in desc_list])

使用BeautifulSoup清洁产品说明的更好方法？

3 个答案: