Question

我已经尝试了整整一周使用antlr构建一个允许我解析电子邮件消息的语法。

我的目标是不将整个电子邮件彻底解析为令牌，而不是解析相关部分。

这是我必须处理的文档格式。 //描述了不属于消息的内联注释：

Subject : [SUBJECT_MARKER] + lorem ipsum...
// marks a message that needs to be parsed.
// Subject marker can be something like "help needed", "action required"

Body: 

// irrelevant text we can ignore, discard or skip
Hi George,
Hope you had a good weekend. Another fluff sentence ...
// end of irrelevant text


// beginning of the SECTION_TYPE_1. SECTION_TYPE_1 marker is "answers below:" 
[SECTION_TYPE_1]

Meaningful text block that needs capturing, made of many sentences: Lorem ipsum ...

[SENTENCE_MARKER] - Sentences that needs to be captured.
[SENTENCE_MARKER] - Sentences that needs to be captured.
[SENTENCE_MARKER] - Sentences that needs to be captured.

[SECTION_END_MARKER] // this is "\n\n"

// SENTENCE_MARKER can be "a)", "b)" or anything that is in the form "[a-zA-Z]')'"
// one important requirement is that this SENTENCE_MARKER matches only inside a section. Either SECTION_TYPE_1 or SECTION_TYPE_2


// alternatively instead of [SECTION_TYPE_1] we can have [SECTION_TYPE_2].
// if we have SECTION_TYPE_1 then try to parse SECTION_TYPE_1 else try to parse SECTION_TYPE_2.enter code here

[SECTION_TYPE_2] // beginning of the section type 1;

Meaningful text bloc that needs capturing. Many sentences Lorem ipsum ...

[SENTENCE_MARKER] - Sentences that needs to be captured.
[SENTENCE_MARKER] - Sentences that needs to be captured.
[SENTENCE_MARKER] - Sentences that needs to be captured.

[SECTION_END_MARKER] // same as above

我面临的问题如下：

我没想出一个在开头跳过文本的好方法消息并在标记出现后才开始应用解析规则被发现了。 SECTION_TYPE_1
捕获部分开头和句子标记之间的部分内的所有文本
在SECTION_END标记后，忽略之后出现的所有文字。

Answer 1

Antlr是一种用于结构化，理想结构明确的文本的解析器。除非您的源消息具有相对明确定义的功能，可靠地标记感兴趣的消息部分，否则Antlr不太可能正常工作。

更好的方法是使用自然语言处理器（NLP）包来识别每个句子或短语的形式和对象，从而识别感兴趣的那些。 Stanford NLP package众所周知（Github）。

更新

必要的语法将采用以下形式：

message : subject ( sec1 | sec2 | fluff )* EOF ;

subject : fluff* SUBJECT_MARKER subText EOL ;
subText : ( word | HWS )+ ;

sec1    : ( SECTION_TYPE_1 content )+ SECTION_END_MARKER     ;
sec2    : ( SECTION_TYPE_2 content )+ SECTION_END_MARKER     ;
content : ( word | ws )+ ;

word    : CHAR+ ;
ws      : ( EOL | HWS )+ ;

fluff   : . ;

SUBJECT_MARKER      : 'marker' ;
SECTION_TYPE_1      : 'text1' ;
SECTION_TYPE_2      : 'text2' ;
SENTENCE_MARKER     : [a-zA-Z0-9] ')' ;

EOL                 : '\r'? '\n';
HWS                 : [ \t] ;
CHAR                : . ;

成功将取决于各种标记的明确程度 - 并且给出了含糊不清的假设。修改语法以明确处理歧义或遵循树遍历/分析阶段来解决。

使用antlr解析电子邮件

1 个答案: