我仍然认为自己是pyparsing的新手。我把2个快速的语法拼凑在一起，并没有成功完成我想做的事情。我试图想出一个看似很简单的语法，但事实证明（至少对我而言）并非如此微不足道。该语言有一个基本定义。它按关键字和正文分类。身体可以跨越多条线。关键字在前20个字符之内的行的开头找到，但以';'结尾（没有引号）。所以我把一个快速的演示程序放在一起，这样我就可以测试几个语法。但是当我尝试使用它们时，它们总是得到第一个关键字，但之后没有。
我已经附加了源代码作为示例和正在发生的输出。虽然这只是测试代码，但出于习惯我做了文档。在下面的示例中，两个关键字现在是;最后;理想情况下，我不希望分号包含在分析符中。
我应该做些什么来使这项工作成功？
from pyparsing import *
def testString(text,grammar):
"""
@summary: perform a test of a grammar
2type text: text
@param text: text buffer for input (a message to be parsed)
@type grammar: MatchFirst or equivalent pyparsing construct
@param grammar: some grammar defined somewhere else
@type pgm: text
@param pgm: typically name of the program, which invoked this function.
@status: 20130802 CODED
"""
print 'Input Text is %s' % text
print 'Grammar is %s' % grammar
tokens = grammar.parseString(text)
print 'After parse string: %s' % tokens
tokens.dump()
tokens.keys()
return tokens
def getText(msgIndex):
"""
@summary: make a text string suitable for parsing
@returns: returns a text buffer
@type msgIndex: int
@param msgIndex: a number corresponding to a text buffer to retrieve
@status: 20130802 CODED
"""
msg = [ """NOW; is the time for a few good ones to come to the aid
of new things to come for it is almost time for
a tornado to strike upon a small hill
when least expected.
lastly; another day progresses and
then we find that which we seek
and finally we will
find our happiness perhaps its closer than 1 or 2 years or not so
""",
'',
]
return msg[msgIndex]
def getGrammar(grammarIndex):
"""
@summary: make a grammar given an index
@type: grammarIndex: int
@param grammarIndex: a number corresponding to the grammar to be retrieved
@Note: a good run will return 2 keys: NOW: and lastly: and each key will have an associated body. The body is all
words and text up to the next keyword or eof which ever is first.
"""
kw = Combine(Word(alphas + nums) + Literal(';'))('KEY')
kw.setDebug(True)
body1 = delimitedList(OneOrMore(Word(alphas + nums)) +~kw)('Body')
body1.setDebug(True)
g1 = OneOrMore(Group(kw + body1))
# ok start defining a new grammar (borrow kw from grammar).
body2 = SkipTo(~kw, include=False)('BODY')
body2.setDebug(True)
g2 = OneOrMore(Group(kw+body2))
grammar = [g1,
g2,
]
return grammar[grammarIndex]
if __name__ == '__main__':
# list indices [ text, grammar ]
tests = {1: [0,0],
2: [0,1],
}
check = tests.keys()
check.sort()
for testno in check:
print 'STARTING Test %d' % testno
text = getText(tests[testno][0])
grammar = getGrammar(tests[testno][1])
tokens = testString(text, grammar)
print 'Tokens found %s' % tokens
print 'ENDING Test %d' % testno
输出如下所示:(使用python 2.7和pyparsing 2.0.1）
STARTING Test 1
Input Text is NOW; is the time for a few good ones to come to the aid
of new things to come for it is almost time for
a tornado to strike upon a small hill
when least expected.
lastly; another day progresses and
then we find that which we seek
and finally we will
find our happiness perhaps its closer than 1 or 2 years or not so
Grammar is {Group:({Combine:({W:(abcd...) ";"}) {{W:(abcd...)}... ~{Combine:({W:(abcd...) ";"})}} [, {{W:(abcd...)}... ~{Combine:({W:(abcd...) ";"})}}]...})}...
Match Combine:({W:(abcd...) ";"}) at loc 0(1,1)
Matched Combine:({W:(abcd...) ";"}) -> ['NOW;']
Match {{W:(abcd...)}... ~{Combine:({W:(abcd...) ";"})}} [, {{W:(abcd...)}... ~{Combine:({W:(abcd...) ";"})}}]... at loc 4(1,5)
Match Combine:({W:(abcd...) ";"}) at loc 161(4,20)
Exception raised:Expected W:(abcd...) (at char 161), (line:4, col:20)
Matched {{W:(abcd...)}... ~{Combine:({W:(abcd...) ";"})}} [, {{W:(abcd...)}... ~{Combine:({W:(abcd...) ";"})}}]... -> ['is', 'the', 'time', 'for', 'a', 'few', 'good', 'ones', 'to', 'come', 'to', 'the', 'aid', 'of', 'new', 'things', 'to', 'come', 'for', 'it', 'is', 'almost', 'time', 'for', 'a', 'tornado', 'to', 'strike', 'upon', 'a', 'small', 'hill', 'when', 'least', 'expected']
Match Combine:({W:(abcd...) ";"}) at loc 161(4,20)
Exception raised:Expected W:(abcd...) (at char 161), (line:4, col:20)
After parse string: [['NOW;', 'is', 'the', 'time', 'for', 'a', 'few', 'good', 'ones', 'to', 'come', 'to', 'the', 'aid', 'of', 'new', 'things', 'to', 'come', 'for', 'it', 'is', 'almost', 'time', 'for', 'a', 'tornado', 'to', 'strike', 'upon', 'a', 'small', 'hill', 'when', 'least', 'expected']]
Tokens found [['NOW;', 'is', 'the', 'time', 'for', 'a', 'few', 'good', 'ones', 'to', 'come', 'to', 'the', 'aid', 'of', 'new', 'things', 'to', 'come', 'for', 'it', 'is', 'almost', 'time', 'for', 'a', 'tornado', 'to', 'strike', 'upon', 'a', 'small', 'hill', 'when', 'least', 'expected']]
ENDING Test 1
STARTING Test 2
Input Text is NOW; is the time for a few good ones to come to the aid
of new things to come for it is almost time for
a tornado to strike upon a small hill
when least expected.
lastly; another day progresses and
then we find that which we seek
and finally we will
find our happiness perhaps its closer than 1 or 2 years or not so
Grammar is {Group:({Combine:({W:(abcd...) ";"}) SkipTo:(~{Combine:({W:(abcd...) ";"})})})}...
Match Combine:({W:(abcd...) ";"}) at loc 0(1,1)
Matched Combine:({W:(abcd...) ";"}) -> ['NOW;']
Match SkipTo:(~{Combine:({W:(abcd...) ";"})}) at loc 4(1,5)
Match Combine:({W:(abcd...) ";"}) at loc 4(1,5)
Exception raised:Expected ";" (at char 7), (line:1, col:8)
Matched SkipTo:(~{Combine:({W:(abcd...) ";"})}) -> ['']
Match Combine:({W:(abcd...) ";"}) at loc 5(1,6)
Exception raised:Expected ";" (at char 7), (line:1, col:8)
After parse string: [['NOW;', '']]
Tokens found [['NOW;', '']]
ENDING Test 2
Process finished with exit code 0
答案 0 :(得分：2)
我很擅长TDD，但是在这里，你的整个测试和替代选择基础设施真正阻碍了语法的位置以及语法的发展。如果我剥离所有额外的机器，我看到你的语法只是：
kw = Combine(Word(alphas + nums) + Literal(';'))('KEY')
body1 = delimitedList(OneOrMore(Word(alphas + nums)) +~kw)('Body')
g1 = OneOrMore(Group(kw + body1))
我看到的第一个问题是你对body1的定义：
body1 = delimitedList(OneOrMore(Word(alphas + nums)) +~kw)('Body')
你在正确的轨道上有一个负向前瞻，但是为了让它在pyparsing中工作，你必须把它放在表达式的开头，而不是结尾。可以把它想象成“在我匹配另一个有效词之前，我将首先排除它是一个关键词。”：
body1 = delimitedList(OneOrMore(~kw + Word(alphas + nums)))('Body')
（顺便说一下，为什么这是一个delimitedList？delimitedList通常是为带有分隔符的真实列表保留的，例如程序函数的逗号分隔参数。所有这一切都接受任何逗号可能会混入身体，应使用标点符号列表更直接地处理。）
这是我的代码的测试版本：
from pyparsing import *
kw = Combine(Word(alphas + nums) + Literal(';'))('KEY')
body1 = OneOrMore(~kw + Word(alphas + nums))('Body')
g1 = OneOrMore(Group(kw + body1))
msg = [ """NOW; is the time for a few good ones to come to the aid
of new things to come for it is almost time for
a tornado to strike upon a small hill
when least expected.
lastly; another day progresses and
then we find that which we seek
and finally we will
find our happiness perhaps its closer than 1 or 2 years or not so
""",
'',
][0]
result = g1.parseString(msg)
# we expect multiple groups, each containing "KEY" and "Body" names,
# so iterate over groups, and dump the contents of each
for res in result:
print res.dump()
我仍然得到与您相同的结果，只是第一个关键字匹配。因此，为了查看断开连接的位置，我使用scanString，它不仅返回匹配的标记，还返回匹配标记的开头和结尾：
result,start,end = next(g1.scanString(msg))
print len(msg),end
这给了我：
320 161
所以我看到我们在一个字符串161中以一个总长度为320的字符串结束，所以我将再添加一个print语句：
print msg[end:end+10]
我得到了：
.
lastly;
正文中的尾随句号是罪魁祸首。如果我从邮件中删除它并再次尝试parseString，我现在得到：
['NOW;', 'is', 'the', 'time', 'for', 'a', 'few', 'good', 'ones', 'to', 'come', 'to', 'the', 'aid', 'of', 'new', 'things', 'to', 'come', 'for', 'it', 'is', 'almost', 'time', 'for', 'a', 'tornado', 'to', 'strike', 'upon', 'a', 'small', 'hill', 'when', 'least', 'expected']
- Body: ['is', 'the', 'time', 'for', 'a', 'few', 'good', 'ones', 'to', 'come', 'to', 'the', 'aid', 'of', 'new', 'things', 'to', 'come', 'for', 'it', 'is', 'almost', 'time', 'for', 'a', 'tornado', 'to', 'strike', 'upon', 'a', 'small', 'hill', 'when', 'least', 'expected']
- KEY: NOW;
['lastly;', 'another', 'day', 'progresses', 'and', 'then', 'we', 'find', 'that', 'which', 'we', 'seek', 'and', 'finally', 'we', 'will', 'find', 'our', 'happiness', 'perhaps', 'its', 'closer', 'than', '1', 'or', '2', 'years', 'or', 'not', 'so']
- Body: ['another', 'day', 'progresses', 'and', 'then', 'we', 'find', 'that', 'which', 'we', 'seek', 'and', 'finally', 'we', 'will', 'find', 'our', 'happiness', 'perhaps', 'its', 'closer', 'than', '1', 'or', '2', 'years', 'or', 'not', 'so']
- KEY: lastly;
如果您想处理标点符号，我建议您添加以下内容：
PUNC = oneOf(". , ? ! : & $")
并将其添加到body1：
body1 = OneOrMore(~kw + (Word(alphas + nums) | PUNC))('Body')