在这里回答了关于在一个城市找到一个城市的问题 用户提供的问题,我开始考虑最佳方式 当您拥有有限的数据集时,在文本中搜索字符串。
in
和find
匹配子字符串,这是不需要的。 Reqular
使用“单词边界”的表达式有效,但速度很慢。该
“标点符号”的方法似乎是一个候选人,但有很多
标点符号characters,既可以出现问题也可以出现
一些以城市的名义(即“圣路易斯”的一个时期)。
Regexps可能是最好的通用解决方案,但我是 好奇,如果可以使用其他技术解决这个问题。
任务是:
使用用户提供的英文文本在美国查找城市 无论如何。
我的代码受http://www.python.org/doc/essays/list2str/
的启发#!/usr/bin/env python
import time
import re
def timing(f, n):
print f.__name__,
r = range(n)
t1 = time.clock()
for i in r:
f(); f(); f(); f(); f(); f(); f(); f(); f(); f()
t2 = time.clock()
print round(t2-t1, 6)
def f0():
'''broken since it finds sub-strings, e.g.
city "Erie" is found in "series"'''
Q = question.upper()
for c in cities:
c = c.upper()
if c in Q:
pass
def f1():
'''slow, but working'''
for c in cities:
re.search('\\b%s\\b' % c, question, re.IGNORECASE)
def f2():
'''broken, same problem as f0()'''
Q = question.upper()
for c in cities:
c = c.upper()
if Q.find(c) > 0:
pass
def f3():
'''remove all punctuation, and then search for " str " '''
Q = question.upper()
punct = ['.', ',', '(', ')', '"', '\n', ' ', ' ', ' ']
for p in punct:
Q = Q.replace(p, ' ')
for c in cities:
c = ' ' + c.upper() + ' '
for p in punct:
c = c.replace(p, ' ')
if c in Q:
pass
with open('cities') as fd:
cities = [line.strip() for line in fd]
with open('question') as fd:
question = fd.readlines()[0]
testfuncs = f0, f1, f2, f3
for f in testfuncs:
print f
timing(f, 20)
在我的旧笔记本电脑上,我得到以下结果
<function f0 at 0xb7730bc4>
f0 0.14
<function f1 at 0xb7730f7c>
f1 10.4
<function f2 at 0xb7730f44>
f2 0.15
<function f3 at 0xb7738684>
f3 0.61
如果有人想要继续我的测试数据,可以找到它 here
答案 0 :(得分:1)
有趣,所有城市的预建正则表达式(即所有城市的一个正则表达式)似乎在表现上获胜。我使用了相同的测试用例,结果如下。
#!/usr/bin/env python
import time
import re
def timing(f, n):
print f.__name__,
r = range(n)
t1 = time.clock()
for i in r:
f(); f(); f(); f(); f(); f(); f(); f(); f(); f()
t2 = time.clock()
print round(t2-t1, 6)
def f0():
'''broken since it finds sub-strings, e.g.
city "Erie" is found in "series"'''
Q = question.upper()
for c in cities:
c = c.upper()
if c in Q:
pass
def f1():
'''slow, but working'''
for c in cities:
re.search('\\b%s\\b' % c, question, re.IGNORECASE)
def f11():
'''Same as f1(). Compiled and searched at runtime.'''
for c in cities:
re.compile('\\b%s\\b' % c, re.IGNORECASE).search(question)
def f12():
'''Building single regex for all cities, and searching using it.'''
regex ="(%s)" % "|".join(re.escape(c) for c in cities)
re.search(regex, question, re.IGNORECASE)
def f13():
'''Using prebuild single regex for all cities to search.'''
re.search(all_cities_regex, question, re.IGNORECASE)
def f14():
'''Building and compiling single regex for all cities, and searching using it.'''
regex = re.compile("(%s)" % "|".join(re.escape(c) for c in cities), re.IGNORECASE)
regex.search(question)
def f15():
'''Searching using prebuild, precompiled regex.'''
precompiled_all.search(question)
def f2():
'''broken, same problem as f0()'''
Q = question.upper()
for c in cities:
c = c.upper()
if Q.find(c) > 0:
pass
def f3():
'''remove all punctuation, and then search for " str " '''
Q = question.upper()
punct = ['.', ',', '(', ')', '"', '\n', ' ', ' ', ' ']
for p in punct:
Q = Q.replace(p, ' ')
for c in cities:
c = ' ' + c.upper() + ' '
for p in punct:
c = c.replace(p, ' ')
if c in Q:
pass
with open('cities') as fd:
cities = [line.strip() for line in fd]
with open('question') as fd:
question = fd.readlines()[0]
all_cities_regex ="(%s)" % "|".join(re.escape(c) for c in cities)
precompiled_all = re.compile("(%s)" % "|".join(re.escape(c) for c in cities), re.IGNORECASE)
testfuncs = f0, f1, f11, f12, f13, f14, f15, f2, f3
for f in testfuncs:
print f
timing(f, 20)
注意:我已经添加了5个函数f11到f15。
这是输出(如我的lappy所示):
<function f0 at 0x259c938>
f0 0.06
<function f1 at 0x259c9b0>
f1 3.81
<function f11 at 0x259ca28>
f11 3.87
<function f12 at 0x259caa0>
f12 0.35
<function f13 at 0x259cb18>
f13 0.2
<function f14 at 0x259cb90>
f14 0.34
<function f15 at 0x259cc08>
f15 0.2
<function f2 at 0x259cc80>
f2 0.06
<function f3 at 0x259ccf8>
f3 0.18
所有城市的预建(f13
)正则表达式(即所有城市的一个正则表达式)都擅长表现。另请注意,预编译此类预构建正则表达式(f15
)并未增加性能。
基于@trutheality和@Thomas的上述评论。
答案 1 :(得分:1)
你的“标点符号”方法的增长:
#!/usr/bin/env python
import time
import re
def timing(f, n):
print f.__name__,
r = range(n)
t1 = time.clock()
for i in r:
f(); f(); f(); f(); f(); f(); f(); f(); f(); f()
t2 = time.clock()
print round(t2-t1, 6)
def f0():
'''broken since it finds sub-strings, e.g.
city "Erie" is found in "series"'''
Q = question.upper()
for c in cities:
c = c.upper()
if c in Q:
pass
def f1():
'''slow, but working'''
for c in cities:
re.search('\\b%s\\b' % c, question, re.IGNORECASE)
def f2():
'''broken, same problem as f0()'''
Q = question.upper()
for c in cities:
c = c.upper()
if Q.find(c) > 0:
pass
def f3():
'''remove all punctuation, and then search for " str " '''
Q = question.upper()
punct = ['.', ',', '(', ')', '"', '\n', ' ', ' ', ' ']
for p in punct:
Q = Q.replace(p, ' ')
for c in cities:
c = ' ' + c.upper() + ' '
for p in punct:
c = c.replace(p, ' ')
if c in Q:
pass
def f4():
'''Single regex which is also broken'''
regex ="(%s)" % "|".join(re.escape(c) for c in cities)
re.search(regex, question, re.IGNORECASE)
def f5():
'''Upgrowth of 'punctiation' approach'''
r = re.compile('\W')
#Additional space is for the case
#when city is at the end of the line
Q = r.sub(' ',''.join([question,' '])).upper()
for c in cities:
C = r.sub(' ',''.join([' ',c,' '])).upper()
if C in Q:
pass
with open('cities') as fd:
cities = [line.strip() for line in fd]
with open('question') as fd:
question = fd.readlines()[0]
testfuncs = f0, f1, f2, f3, f4, f5
for f in testfuncs:
print f
timing(f, 20)
速度非常快:
<function f0 at 0x01F9B470>
f0 0.092498
<function f1 at 0x01F9B530>
f1 6.48321
<function f2 at 0x01F9B870>
f2 0.101243
<function f3 at 0x01F9B3F0>
f3 0.304404
<function f4 at 0x01F9B4F0>
f4 0.671799
<function f5 at 0x01F9B570>
f5 0.278714