Question

我正在用python编写c / c ++词法分析器。

它还有很多工作要做，但到目前为止，我还处于困境。我想使用RegEx模式检查变量名称：

（？：\ w + \ s +）（[a-zA-Z _] [a-zA-Z0-9] +）/ g

尽管该模式在regexr上正常运行

我要执行此操作的代码是：

摘要：

用于检查字符串是否与模式匹配的代码：

regex = re.compile(r'(?:\w+\s+)([a-zA-Z_][a-zA-Z0-9]+)\b')
if re.search(regex, token) == True: #if token matches the pattern
            print(token + ' : Variable Name')

import re


#dictionaries:
#1 operators
operators = {'=': 'Assignment',
    '+': 'Additon',
    '-' : 'Substraction',
    '/' : 'Division',
    '*': 'Multiplication',
    '++' : 'increment',
    '--' : 'Decrement',
    '<': 'Lower Than',
    '>': 'Greater Than'}
optr_keys = operators.keys()

#2 keywords
keywords = {'int': 'Integer Datatype Indicator',
    'float': 'Floating Point Datatype Indicator',
    'char': 'Character Datatype Indicator',
    'long': 'Long Int Datatype Indicator',
    'double': 'Double Datatype Indicator',
    'short': 'Short Integer Datatype Indicator',
    'unsigned': 'Unsigned Integer Datatype Indicator',
    'void': 'Void Datatype Indicator',
    'struct': 'Structure Datatype Indicator',
    'return': 'Return',
    'if': 'Condition If Keyword',
    'else': 'Condition Else Keyword',
    'while': 'While Loop Indicator',
    'do': 'Do While Loop Indicator',
    'break': 'Break Out Keyword',
    'continue': 'Continue Keyword',
    'switch': 'Switch Keyword',
    'case': 'Case Keyword',
    'sizeof': 'Variable Size Indicator',
    'typedef': 'Function Type Indicator',
    'static': 'Static Type Keyword',
    'goto': 'Go To line Keyword',
    '#include': 'Header Include Indicator'
    }
keyword_keys = keywords.keys()

#3 delimiters
delimiters = {';':'Line Terminator (Semicolon)',
    ' ': 'Single Empty Space'}
delimiter_keys = delimiters.keys()

#4 comment indicators
comments = {r'//' : 'Single Line Comment',
    r'/*' : 'Multiline Comment Start',
    r'*/' : 'Multiline Comment End',
    '/**/' : 'Empty Multiline comment'}
comment_keys = comments.keys()

#5 builtin header files
header_files = {'<stdio.h>': 'Standard Input Output Header',
    '<string.h>':'String Manipulation Library'}
header_keys = header_files.keys()

#6 blocks
blocks = {'{' : 'Blocked Statement Body Open',
    '}':'Blocked Statement Body Closed'}
blocks_keys = blocks.keys()

#7 predefined functions
builtin_functions = {'printf':'Prints To Console',
    'cout': 'Standard Output Function',
    'cin': 'Standard Input Function'}
builtinfunc_keys = builtin_functions.keys()

#8 numbers
numbers = {'0': 'Zero',
    '1': 'One',
    '2': 'Two',
    '3': 'Three',
    '4': 'Four',
    '5': 'Five',
    '6': 'Six',
    '7': 'Se7en',
    '8': 'Eight',
    '9': 'Nine'}
numbers_keys = numbers.keys()

count = 0
cfile = '/some/path/to/sample/file.c'
f = open(cfile, 'r').read()
lines = f.split('\n')
regex = re.compile(r'(?:\w+\s+)([a-zA-Z_][a-zA-Z0-9]+)\b')
for line in lines:
    count = count + 1
    print('\n\n###Line Number', str(count) + '\n')
    tokens = line.split(' ')
    print('Tokens Are ', tokens)

    for token in tokens:
        if '\n' in token:
            position = token.find('\n')
            token=token[:position]

        if token in optr_keys:
            print(token, ' : Operator => ', operators[token])

        elif token in keyword_keys:
            print(token, ' : Keyword => ', keywords[token])

        elif token in comment_keys:
            print(token + ' : Comment => ', comments[token])

        elif '.h' in token:
            print(token + ' : Header File => ', header_files[token])

        elif token in blocks_keys:
            print(token + ' : Block Indicator => ', blocks[token])

        elif token in builtinfunc_keys:
            print(token + ' : Built-in Function => ', builtin_functions[token])

        elif token in numbers:
            print(token + ' : Numbers => ', numbers_keys[token])

        else:
            if bool(re.search(regex, token)) == True: #if token matches the pattern
                print(token + ' : Variable Name')

示例输出：

###Line Number 1

Tokens Are  ['#include', '<stdio.h>', '//', 'This', 'is', 'a', 'header', 'file']
#include  : Keyword =>  Header Include Indicator <stdio.h> : Header File =>  Standard Input Output Header // : Comment =>  Single Line Comment


###Line Number 2

Tokens Are  ['int', 'main()'] int  : Keyword =>  Integer Datatype Indicator


###Line Number 3

Tokens Are  ['{'] { : Block Indicator =>  Blocked Statement Body Open


###Line Number 4

Tokens Are  ['', '', '', '', 'int', 'a;'] int  : Keyword =>  Integer Datatype Indicator


###Line Number 5

Tokens Are  ['', '', '', '', 'a', '=', '10;']
=  : Operator =>  Assignment


###Line Number 6

Tokens Are  ['', '', '', '', 'printf("The', 'value', 'of', 'a', 'is', '%d', '",a);']


###Line Number 7

Tokens Are  ['', '', '', '', 'return', '0;'] return  : Keyword =>  Return


###Line Number 8

Tokens Are  ['}'] } : Block Indicator =>  Blocked Statement Body Closed


###Line Number 9

Tokens Are  ['']

我希望代码在输出中包含变量名，但是由于匹配过程失败，因此它只会忽略它们。我想我尝试匹配标记/字符串的方式出了问题。

Answer 1

问题出在我的正则表达式本身上。

以下模式有效：

[a-zA-Z _] [a-zA-Z0-9 _]

积分：Jon Guiton

如何检查令牌是否与RegEx模式匹配？

摘要：

1 个答案: