Question

当我在Python中处理HTML代码时，由于特殊字符，我必须使用以下代码。

line = string.replace(line, "&quot;", "\"")
line = string.replace(line, "&apos;", "'")
line = string.replace(line, "&amp;", "&")
line = string.replace(line, "&lt;", "<")
line = string.replace(line, "&gt;", ">")
line = string.replace(line, "&laquo;", "<<")
line = string.replace(line, "&raquo;", ">>")
line = string.replace(line, "&#039;", "'")
line = string.replace(line, "&#8220;", "\"")
line = string.replace(line, "&#8221;", "\"")
line = string.replace(line, "&#8216;", "\'")
line = string.replace(line, "&#8217;", "\'")
line = string.replace(line, "&#9632;", "")
line = string.replace(line, "&#8226;", "-")

似乎还有更多我需要替换的特殊字符。你知道如何使这段代码更优雅吗？

谢谢

Answer 1

REPLACEMENTS = [
    ("&quot;", "\""),
    ("&apos;", "'"),
    ...
    ]
for entity, replacement in REPLACEMENTS:
    line = line.replace(entity, replacement)

请注意，string.replace仅作为str / unicode个对象的方法提供。

更好的是，请查看this question！

你的问题标题要求不同的东西：优化，即让它运行得更快。这是一个完全不同的问题，需要更多的工作。

Answer 2

这是我写回来解码HTML实体的一些代码。请注意，它适用于Python 2.x，因此它也会从str解码为unicode：如果您使用的是现代Python，则可以删除该位。我认为它处理任何命名实体，十进制和十六进制实体。出于某种原因'''不在Python的命名实体字典中，所以我先复制它并添加缺少的实体：

from htmlentitydefs import name2codepoint
name2codepoint = name2codepoint.copy()
name2codepoint['apos']=ord("'")

EntityPattern = re.compile('&(?:#(\d+)|(?:#x([\da-fA-F]+))|([a-zA-Z]+));')
def decodeEntities(s, encoding='utf-8'):
    def unescape(match):
        code = match.group(1)
        if code:
            return unichr(int(code, 10))
        else:
            code = match.group(2)
            if code:
                return unichr(int(code, 16))
            else:
                code = match.group(3)
                if code in name2codepoint:
                    return unichr(name2codepoint[code])
        return match.group(0)

    if isinstance(s, str):
        s = s.decode(encoding)
    return EntityPattern.sub(unescape, s)

Answer 3

优化

REPL_tu = (("&quot;", "\"") , ("&apos;", "'") , ("&amp;", "&") ,
           ("&lt;", "<") , ("&gt;", ">") ,
           ("&laquo;", "<<") , ("&raquo;", ">>") ,
           ("&#039;", "'") ,
           ("&#8220;", "\"") , ("&#8221;", "\"") ,
           ("&#8216;", "\'") , ("&#8217;", "\'") ,
           ("&#9632;", "") , ("&#8226;", "-")     )

def repl(mat, d = dict(REPL_tu)):
    return d[mat.group()]

import re
regx = re.compile('|'.join(a for a,b in REPL_tu))

line = 'A tag &lt;bidi&gt; has a &quot;weird&#8220;&#8226;&apos;content&apos;'
modline = regx.sub(repl,line)
print 'Exemple:\n\n'+line+'\n'+modline 








from urllib import urlopen

print '\n-----------------------------------------\nDownloading a web source:\n'
sock = urlopen('http://www.mythicalcreaturesworld.com/greek-mythology/monsters/python-the-serpent-of-delphi-%E2%80%93-python-the-guardian-dragon-and-apollo/')
html_source = sock.read()
sock.close()

from time import clock

n = 100

te = clock()
for i in xrange(n):
    res1 = html_source
    res1 = regx.sub(repl,res1)
print 'with regex  ',clock()-te,'seconds'


te = clock()
for i in xrange(n):
    res2 = html_source
    for entity, replacement in REPL_tu:
        res2 = res2.replace(entity, replacement)
print 'with replace',clock()-te,'seconds'

print res1==res2

结果

Exemple:

A tag &lt;bidi&gt; has a &quot;weird&#8220;&#8226;&apos;content&apos;
A tag <bidi> has a "weird"-'content'

-----------------------------------------
Downloading a web source:

with regex   0.097578323502 seconds
with replace 0.213866846205 seconds
True

使string.replace语句的序列更具可读性

3 个答案: