get_text()具有UnicodeEncodeError

时间:2012-04-22 05:30:20

标签: python unicode ascii beautifulsoup

我有以下HTML:

<div class="dialog">
<div class="title title-with-sort-row">
    <h2>Description</h2>
    <div class="dialog-search-sort-bar">
    </div>
</div>
<div class="content"><div style="margin-right: 20px; margin-left: 30px;">
    <span class="description2">
        With “Antonia Polygon – Standard”, you have a figure that is unique in the Poser community. 
        She is made available under a Creative Commons License that gives endless opportunities for further development. 
        This figure was developed by a group of talented members of the Poser community in a thirty-month effort. 
        The result is a figure that has very good bending and morphing behavior.
        <br />
    </span>
</div>
</div>

我需要从class="dialog"的几个div中找到这个div,然后在span class="description2"中删除该文本。

当我使用代码时:

description = soup.find(text = re.compile('Description'))
if description != None:
    someEl = description.parent
    parent1 = someEl.parent
    parent2 = parent1.parent
    description = parent2.find('span', {'class' : 'description2'})
    print 'Description: ' + str(description)

我明白了:

<span class="description2">
    With “Antonia Polygon – Standard”, you have a figure that is unique in the Poser community. 
    She is made available under a Creative Commons License that gives endless opportunities for further development. 
    This figure was developed by a group of talented members of the Poser community in a thirty-month effort. 
    The result is a figure that has very good bending and morphing behavior.
    <br/>
</span>

如果我尝试获取文本,而不使用HTML&amp;非ASCII字符,使用

description = description.get_text()

我得到 (UnicodeEncodeError): 'ascii' codex can't encode character u'\x93'

如何将此HTML块转换为直接ascii?

1 个答案:

答案 0 :(得分:2)

#!/usr/bin/env python
# -*- coding: utf-8 -*-

foo = u'With “Antonia Polygon – Standard”, you have a figure that is unique in the Poser community.She is made available under a Creative Commons License that gives endless opportunities for further development. This figure was developed by a group of talented members of the Poser community in a thirty-month effort. The result is a figure that has very good bending and morphing behavior.'

print foo.encode('ascii', 'ignore')

需要注意的三件事。

首先是encode方法的'ignore'参数。它指示方法删除不在所选编码范围内的字符(在这种情况下,ascii是安全的)。

其次,我们通过在字符串前加u来明确地将foo设置为unicode。

第三个是显式文件编码指令:# -*- coding: utf8 -*-

另外,如果你在这个答案的评论中没有读到Daenyth的非常好的观点那么你就是一个愚蠢的土块。如果要在HTML / XML中使用输出,xmlcharrefreplace可用于代替ignore以上的正义。