如何改善pyparsing启动时间?

时间:2011-10-04 07:48:17

标签: performance pyparsing

我用pyparsing编写了一个大的(对我来说)语法来解析H248消息,这是一种文本格式,有大约500行的pypasing结构。我发现平均性能是可以接受的,如果我给解析器提供许多要解析的消息(我的电脑上每条消息大约半秒钟)。

但是我的解析器的基本用法是解析一条消息,然后转储掉某些部分,然后我把它写成一个独立的程序。现在我的x86 pc上花了12秒,(在共享的solaria服务器上1m12s!)来解析第一条消息。所以我想知道是否有任何技术可以绕过这个启动时间?我尝试过psyco,它没有帮助启动时间。

我现在要做的是编写一个长生活服务器导入解析器并接受解析请求,然后另一个独立程序(经常被触发)与该服务器进程交互,从而获得快速响应效果。这样,我就失去了将解析结果用作python对象的机会。我想知道是否有更好的方法来达到我的目的?

顺便说一句,如果我在12秒之前手动中断我的解析器,那么堆栈跟踪显示它正在递归地调用streamline()。

谢谢! 凯文

    from pyparsing import *
    ALPHA = alphas
    DIGIT = nums
    HEXDIG = "0123456789ABCDEF"
    SafeChar = alphanums +  "+-&!_'?@^`~*$\\()%+."
    RestChar = ";[]{}:,#<>="
    COLON = ":"
    DOT = "."
    SLASH = "/"
    DQUOTE = '"'
    SP = " "
    HTAB = '\\0x09'
    HTAB = ''
    EOL = Suppress(lineEnd)
    WSP = SP + HTAB 
    COMMENT = Word(";",  SafeChar + RestChar + WSP + '"') + EOL
    LWSP = Suppress(Word(WSP))
    nonEscapeChar = "}" + srange("[\\0x01-\\0x7c]") #+ srange("[\\0x7E-\\0xFF]")
    nonEscapeChar = srange("[\\0x01-\\0x7c]") #+ srange("[\\0x7E-\\0xFF]")
    octetString = Word(nonEscapeChar)
    quotedString = dblQuotedString
    UINT16 = Word(DIGIT, min=1, max=5) 
    UINT32 = Word(DIGIT, min=1, max=10) 
    NAME = Word(ALPHA, alphanums + "_", min=1, max=64)
    VALUE = quotedString | Word(SafeChar) #| srange("[\\0x80-\\0xFF]")
    EQUAL = Suppress("=")
    LBRKT = Suppress( "{" )
    RBRKT = Suppress( "}" )
    COMMA = Suppress(",")
    SEP = ((WSP | EOL | COMMENT) + LWSP)
    AddToken = Suppress(Literal("Add")| "A")
    AndAUDITSelectToken = Suppress(Literal("ANDLgc"))
    AuditToken = Suppress(Literal("Audit")| "AT")
    AuditCapToken = Suppress(Literal("AuditCapability")| "AC")
    AuditValueToken = Suppress(Literal("AuditValue")| "AV")
    AuthToken = Suppress(Literal("Authentication")| "AU")
    BothToken = Suppress(Literal("Both")| "B")
    BothwayToken = Suppress(Literal("Bothway")| "BW")
    BriefToken = Suppress(Literal("Brief")| "BR")
    BufferToken = Suppress(Literal("Buffer")| "BF")
    CtxToken = Suppress(Literal("Context")| "C")
    ContextAuditToken = Suppress(Literal("ContextAudit")| "CA")
    ContextAttrToken = Suppress(Literal("ContextAttr")| "CT")
    ContextListToken = Suppress(Literal("ContextList")| "CLT")
    DigitMapToken = Suppress(Literal("DigitMap")| "DM")
    DirectionToken = Suppress(Literal("SPADirection")| "SPADI")
    DisconnectedToken = Suppress(Literal("Disconnected")| "DC")
    DelayToken = Suppress(Literal("Delay")| "DL")
    DurationToken = Suppress(Literal("Duration")| "DR")
    EmbedToken = Suppress(Literal("Embed")| "EM")
    EmergencyToken = Suppress(Literal("Emergency")| "EG")
    EmergencyOffToken = Suppress(Literal("EmergencyOff")| "EGO")
    EmergencyValueToken = Suppress(Literal("EmergencyValue")| "EGV")
    ErrorToken = Suppress(Literal("Error")| "ER")
    EventBufferToken = Suppress(Literal("EventBuffer")| "EB")
    EventsToken = Suppress(Literal("Events")| "E")
    ExternalToken = Suppress(Literal("External")| "EX")
    FailoverToken = Suppress(Literal("Failover")| "FL")
    ForcedToken = Suppress(Literal("Forced")| "FO")
    GracefulToken = Suppress(Literal("Graceful")| "GR")
    H221Token = Suppress(Literal("H221"))
    H223Token = Suppress(Literal("H223"))
    H226Token = Suppress(Literal("H226"))
    HandOffToken = Suppress(Literal("HandOff")| "HO")
    IEPSToken = Suppress(Literal("IEPSCall")| "IEPS")
    ImmAckRequiredToken = Suppress(Literal("ImmAckRequired")| "IA")
    InternalToken = Suppress(Literal("Internal")| "IT")
    IntsigDelayToken = Suppress(Literal("Intersignal")| "SPAIS")
    IsolateToken = Suppress(Literal("Isolate")| "IS")
    InSvcToken = Suppress(Literal("InService")| "IV")
    InterruptByEventToken = Suppress(Literal("IntByEvent")| "IBE")
    InterruptByNewSignalsDescrToken = Suppress(Literal("IntBySigDescr")| "IBS")
    IterationToken = Suppress(Literal("Iteration")| "IR")
    KeepActiveToken = Suppress(Literal("KeepActive")| "KA")
    LocalToken = Suppress(Literal("Local")| "L")
    LocalControlToken = Suppress(Literal("LocalControl")| "O")
    LockStepToken = Suppress(Literal("LockStep")| "SP")
    MediaToken = Suppress(Literal("Media")| "M")
    MegacopToken = Suppress(Literal("MEGACO")| "!")
    MessageSegmentToken = Suppress(Literal("Segment")| "SM")
    MethodToken = Suppress(Literal("Method")| "MT")
    MgcIdToken = Suppress(Literal("MgcIdToTry")| "MG")
    ModeToken = Suppress(Literal("Mode")| "MO")
    ModifyToken = Suppress(Literal("Modify")| "MF")
    ModemToken = Suppress(Literal("Modem")| "MD")
    MoveToken = Suppress(Literal("Move")| "MV")
    MTPToken = Suppress(Literal("MTP"))
    MuxToken = Suppress(Literal("Mux")| "MX")
    NeverNotifyToken = Suppress(Literal("NeverNotify")| "NBNN")
    NotifyToken = Suppress(Literal("Notify")| "N")
    NotifyCompletionToken = Suppress(Literal("NotifyCompletion")| "NC")
    NotifyImmediateToken = Suppress(Literal("ImmediateNotify")| "NBIN")
    NotifyRegulatedToken = Suppress(Literal("RegulatedNotify")| "NBRN")
    Nx64kToken = Suppress(Literal("Nx64Kservice")| "N64")
    ObservedEventsToken = Suppress(Literal("ObservedEvents")| "OE")
    OnewayToken = Suppress(Literal("Oneway")| "OW")
    OnewayBothToken = Suppress(Literal("OnewayBoth")| "OWB")
    OnewayExternalToken = Suppress(Literal("OnewayExternal")| "OWE")
    OnOffToken = Suppress(Literal("OnOff")| "OO")
    OrAUDITselectToken = Suppress(Literal("ORLgc"))
    OtherReasonToken = Suppress(Literal("OtherReason")| "OR")
    OutOfSvcToken = Suppress(Literal("OutOfService")| "OS")
    PackagesToken = Suppress(Literal("Packages")| "PG")
    #...many literal tokens 
    TestToken = Suppress(Literal("Test")| "TE")
    TimeOutToken = Suppress(Literal("TimeOut")| "TO")
    TopologyToken = Suppress(Literal("Topology") | "TP")
    TransToken = Suppress(Literal("Transaction") | "T")
    ResponseAckToken = Suppress(Literal("TransactionResponseAck")| "K")
    V18Token = Suppress(Literal("V18"))
    V22Token = Suppress(Literal("V22"))
    V22bisToken = Suppress(Literal("V22b"))
    V32Token = Suppress(Literal("V32"))
    V32bisToken = Suppress(Literal("V32b"))
    V34Token = Suppress(Literal("V34"))
    V76Token = Suppress(Literal("V76"))
    V90Token = Suppress(Literal("V90"))
    V91Token = Suppress(Literal("V91"))
    VersionToken = Suppress(Literal("Version") | "V")
    RSBRKT=Suppress("]")
    LSBRKT=Suppress("]")
    INEQUAL=Suppress(LWSP+(Literal(">")|"<"|"#")+LWSP)("INEQUAL")
    ContextID=Combine((UINT32|"*"|"-"|"$"))("ContextID")
    ItemID=NAME("ItemID")
    PackageName=NAME("PackageName")
    StreamID=UINT16("StreamID")
    Time=(Word(DIGIT, exact=8))("Time")
    Date=(Word(DIGIT, exact=8))("Date")
    RequestID=(UINT32|"*")("RequestID")
    digitMapName=NAME("digitMapName")
    sigParameterName=NAME("sigParameterName")
    signalListId=UINT16("signalListId")
    pathDomainName=Group(Word(alphanums+"*",alphanums+"-"+"*"+".",max=64))("pathDomainName")
    pathNAME=Combine(Optional("*")+NAME+Word("/"+"*"+alphanums+"_"+"$")+Optional("@"+pathDomainName))("pathNAME")
    pkgdName=Combine((PackageName+SLASH+ItemID)|(PackageName+SLASH+"*")|("*"+SLASH+"*"))
    alternativeValue=Combine((VALUE|LSBRKT+delimitedList(VALUE, COMMA)+RSBRKT|LBRKT+delimitedList(VALUE, COMMA) +RBRKT|LSBRKT+VALUE+COLON+VALUE+RSBRKT))("alternativeValue")
    parmValue=((EQUAL+alternativeValue|INEQUAL+VALUE))("parmValue")
    propertyParm=Group(pkgdName+parmValue)("propertyParm")
    extensionParameter=Group(Literal("X")+(Literal("-")|"+")+Word(ALPHA+DIGIT,min=1,max=6))("extensionParameter")
    contextIdList=Group(ContextListToken+EQUAL+LBRKT+delimitedList(ContextID, COMMA) +RBRKT)("contextIdList")
    contextAttrDescriptor=Group(ContextAttrToken+LBRKT+(contextIdList|delimitedList(propertyParm, COMMA))+RBRKT)("contextAttr")
    emergencyValue=Group(EmergencyValueToken+EQUAL+(EmergencyToken|EmergencyOffToken))("emergencyValue")
    iepsValue=Group(IEPSToken+EQUAL+(Literal("ON")|"OFF"))("iepsValue")
    priority=Group(PriorityToken+EQUAL+UINT16)("priority")
    topologyDirection=Group(BothwayToken|IsolateToken|OnewayToken|OnewayExternalToken|OnewayBothToken)("topologyDirection")
    TerminationID=(Literal("ROOT")|pathNAME|"$"|"*")("TerminationID")
    terminationB=(TerminationID)("terminationB")
    terminationA=(TerminationID)("terminationA")
    eventStream=Group(StreamToken+EQUAL+StreamID)("eventStream")
    topologyTriple=Group(terminationA+COMMA+terminationB+COMMA+topologyDirection+Optional(COMMA+eventStream))("topologyTriple")
    topologyDescriptor=Group(TopologyToken+LBRKT+delimitedList(topologyTriple, COMMA)+RBRKT)("topology")
    statisticsParameter=Group(pkgdName+Optional(EQUAL+VALUE|(LSBRKT+delimitedList(VALUE, COMMA)+RSBRKT)))("statisticsParameter")
    statisticsDescriptor=Group(StatsToken+LBRKT+delimitedList(statisticsParameter, COMMA)+RBRKT)("statistics")
    TimeStamp=Group(Date+"T"+Time)("TimeStamp")
    packagesItem=Group(NAME+"-"+UINT16)("packagesItem")
    Version=Group(Word(DIGIT,min=1,max=2))("Version")
    packagesDescriptor=Group(PackagesToken+LBRKT+delimitedList(packagesItem, COMMA)+RBRKT)("packages")
    extension=Group(extensionParameter+parmValue)("extension")
    serviceChangeVersion=Group(VersionToken+EQUAL+Version)("serviceChangeVersion")
    serviceChangeProfile=Group(ProfileToken+EQUAL+NAME+SLASH+Version)("serviceChangeProfile")
    deviceName=Group(pathNAME)("deviceName")
    TransactionID=UINT32("TransactionID")
    ErrorCode=Word(DIGIT,min=1,max=4)("ErrorCode")
    mtpAddress=Group(MTPToken+LBRKT+Word(HEXDIG,min=4,max=8)+RBRKT)("mtpAddress")
    portNumber=Group(UINT16)("portNumber")
    hex4=Group(Word(HEXDIG,min=1,max=4))("hex4")
    hexseq=Group(hex4+ZeroOrMore(":"+hex4))("hexseq")
    hexpart=Group(hexseq+"::"+Optional(hexseq)|"::"+Optional(hexseq)|hexseq)("hexpart")
    V4hex=Group(Word(DIGIT,min=1,max=3))("V4hex")
    IPv4address=Group(V4hex+DOT+V4hex+DOT+V4hex+DOT+V4hex)("IPv4address")
    IPv6address=Group(hexpart+Optional(":"+IPv4address))("IPv6address")
    domainAddress=Group(Literal("[")+(IPv4address|IPv6address)+"]")("domainAddress")
    domainName=Group("<"+Word(ALPHA+DIGIT,ALPHA+DIGIT+"-"+".",max=64)+">")("domainName")
    mId=(((domainAddress|domainName)+Optional(":"+portNumber))|mtpAddress|deviceName)("mId")
    sigIntsigDelay=Group(IntsigDelayToken+EQUAL+UINT16)("sigIntsigDelay")
    sigRequestID=(RequestIDToken+EQUAL+RequestID)("sigRequestID")
    direction=Group(ExternalToken|InternalToken|BothToken)("direction")
    sigDirection=Group(DirectionToken+EQUAL+direction)("sigDirection")
    sigDuration=Group(DurationToken+EQUAL+UINT16)("sigDuration")
    signalType=Group((OnOffToken|TimeOutToken|BriefToken))("signalType")
    sigSignalType=Group(SignalTypeToken+EQUAL+signalType)("sigSignalType")
    sigOther=Group(sigParameterName+parmValue)("sigOther")
    sigStream=Group(StreamToken+EQUAL+StreamID)("sigStream")
    signalName=Group(pkgdName)("signalName")
    notificationReason=Group(TimeOutToken|InterruptByEventToken|InterruptByNewSignalsDescrToken|OtherReasonToken|IterationToken)("notificationReason")
    notifyCompletion=Group(NotifyCompletionToken+EQUAL+(LBRKT+delimitedList(notificationReason, COMMA)+RBRKT))("notifyCompletion")
    sigParameter=Group(sigStream|sigSignalType|sigDuration|sigOther|notifyCompletion|KeepActiveToken|sigDirection|sigRequestID|sigIntsigDelay)("sigParameter")
    signalRequest=Group(signalName+Optional(LBRKT+delimitedList(sigParameter, COMMA)+RBRKT))("signalRequest")
    signalListParm=Group(signalRequest)("signalListParm")
    signalList=Group(SignalListToken+EQUAL+signalListId+LBRKT+delimitedList(signalListParm,COMMA)+RBRKT)("signalList")
    signalParm=Group(signalList|signalRequest)("signalParm")
    signalsDescriptor=Group(SignalsToken+Optional(LBRKT+delimitedList(signalParm,COMMA)+RBRKT))("signals")
    serviceChangeMgcId=(MgcIdToken+EQUAL+mId)("serviceChangeMgcId")
    serviceChangeAddress=Group(ServiceChangeAddressToken+EQUAL+(mId|portNumber))("serviceChangeAddress")
    serviceChangeDelay=Group(DelayToken+EQUAL+UINT32)("serviceChangeDelay")
    serviceChangeReason=Group(ReasonToken+EQUAL+VALUE)("serviceChangeReason")
    serviceChangeMethod=Group(MethodToken+EQUAL+(FailoverToken|ForcedToken|GracefulToken|RestartToken|DisconnectedToken|HandOffToken|extensionParameter))("serviceChangeMethod")
    servChgReplyParm=Group(serviceChangeAddress|serviceChangeMgcId|serviceChangeProfile|serviceChangeVersion|TimeStamp)("servChgReplyParm")
    serviceChangeReplyDescriptor=Group(ServicesToken+LBRKT+delimitedList(servChgReplyParm, COMMA)+RBRKT)("serviceChangeReply")
    auditReturnItem=Group((MuxToken|ModemToken|MediaToken|DigitMapToken|StatsToken|ObservedEventsToken|PackagesToken))("auditReturnItem")
    indAudpackagesDescriptor=Group(PackagesToken+LBRKT+packagesItem+RBRKT)("indAudpackages")
    indAudstatisticsDescriptor=Group(StatsToken+LBRKT+pkgdName+RBRKT)("indAudstatistics")
    indAuddigitMapDescriptor=Group(DigitMapToken+EQUAL+(digitMapName))("indAuddigitMap")
    indAudsignalRequestParm=Group(sigStream|sigRequestID)("indAudsignalRequestParm")
    indAudsignalRequest=Group(signalName+Optional(LBRKT+delimitedList(indAudsignalRequestParm, COMMA)+RBRKT))("indAudsignalRequest")
    indAudsignalListParm=Group(indAudsignalRequest)("indAudsignalListParm")
    indAudsignalList=Group(SignalListToken+EQUAL+signalListId+Optional(LBRKT+indAudsignalListParm+RBRKT))("indAudsignalList")
    indAudsignalParm=Group(indAudsignalList|indAudsignalRequest)("indAudsignalParm")
    indAudsignalsDescriptor=Group(SignalsToken+LBRKT+Optional(indAudsignalParm)+RBRKT)("indAudsignals")
    indAudrequestedEvent=Group(pkgdName)("indAudrequestedEvent")
    indAudeventsDescriptor=Group(EventsToken+Optional(EQUAL+RequestID)+LBRKT+indAudrequestedEvent+RBRKT)("indAudevents")
    eventParameterName=Group(NAME)("eventParameterName")
    eventOther=Group(eventParameterName+parmValue)("eventOther")
    indAudeventSpecParameter=Group(eventStream|eventParameterName)("indAudeventSpecParameter")
    indAudeventSpec=Group(pkgdName+Optional(LBRKT+indAudeventSpecParameter+RBRKT))("indAudeventSpec")
    indAudeventBufferDescriptor=Group(EventBufferToken+LBRKT+indAudeventSpec+RBRKT)("indAudeventBuffer")
    serviceStatesValue=Group((TestToken|OutOfSvcToken|InSvcToken))("serviceStatesValue")
    indAudterminationStateParm=Group(pkgdName|propertyParm|ServiceStatesToken+Optional((EQUAL|INEQUAL)+serviceStatesValue)|BufferToken)("indAudterminationStateParm")
    indAudterminationStateDescriptor=Group(TerminationStateToken+LBRKT+indAudterminationStateParm+RBRKT)("indAudterminationState")
    streamModes=(SendonlyToken|RecvonlyToken|SendrecvToken|InactiveToken|LoopbackToken)("Mode")
    indAudlocalParm=Group(ModeToken+Optional((EQUAL|INEQUAL)+streamModes)|pkgdName|propertyParm|ReservedValueToken|ReservedGroupToken)("indAudlocalParm")
    indAudlocalControlDescriptor=Group(LocalControlToken+LBRKT+delimitedList(indAudlocalParm, COMMA)+RBRKT)("indAudlocalControl")
    indAudstreamParm=Forward()
    indAudstreamDescriptor=Group(StreamToken+EQUAL+StreamID+LBRKT+indAudstreamParm+RBRKT)("indAudstream")
    indAudlocalDescriptor=Group(LocalToken+LBRKT+octetString+RBRKT)("indAudlocal")
    indAudremoteDescriptor=Group(RemoteToken+LBRKT+octetString+RBRKT)("indAudremote")
    indAudstreamParm<<(indAudlocalControlDescriptor|indAudstatisticsDescriptor|indAudremoteDescriptor|indAudlocalDescriptor)
    indAudmediaParm=indAudstreamParm|indAudstreamDescriptor|indAudterminationStateDescriptor
    indAudmediaDescriptor=Group(MediaToken+LBRKT+delimitedList(indAudmediaParm, COMMA)+RBRKT)("indAudmedia")
    indAudauditReturnParameter=indAudmediaDescriptor|indAudeventsDescriptor|indAudsignalsDescriptor|indAuddigitMapDescriptor|indAudeventBufferDescriptor|indAudstatisticsDescriptor|indAudpackagesDescriptor
    indAudterminationAudit=Group(delimitedList(indAudauditReturnParameter, COMMA))("indAudterminationAudit")
    auditItem=Group(auditReturnItem|SignalsToken|EventBufferToken|EventsToken|indAudterminationAudit)("auditItem")
    serviceChangeParm=Group(serviceChangeMethod|serviceChangeReason|serviceChangeDelay|serviceChangeAddress|serviceChangeProfile|extension|TimeStamp|serviceChangeMgcId|serviceChangeVersion|ServiceChangeIncompleteToken|auditItem)("serviceChangeParm")
    serviceChangeDescriptor=Group(ServicesToken+LBRKT+delimitedList(serviceChangeParm,COMMA)+RBRKT)("serviceChange")
    digitMapLetter=DIGIT+srange("[\\0x41-\\0x4B]")+srange("[\\0x61-\\0x6B]")+"L"+"S"+"T"+"Z"
    digitLetter=Group(Word(DIGIT+"-"+DIGIT)|Word(digitMapLetter))("digitLetter")
    digitMapRange=Group((Literal("x")|(LWSP+"["+LWSP+digitLetter+LWSP+"]"+LWSP)))("digitMapRange")
    digitPosition=Group(digitMapLetter|digitMapRange)("digitPosition")
    digitStringElement=Group(digitPosition+Optional(DOT))("digitStringElement")
    digitString=Group(OneOrMore(digitStringElement))("digitString")
    digitStringList=Group(digitString+ZeroOrMore(LWSP+"|"+LWSP+digitString))("digitStringList")
    digitMap=Group((digitString|LWSP+"("+LWSP+digitStringList+LWSP+")"+LWSP))("digitMap")
    Timer=Group(Word(DIGIT,min=1,max=2))("Timer")
    digitMapValue=Group(Optional("T"+COLON+Timer+COMMA)+Optional("S"+COLON+Timer+COMMA)+Optional("L"+COLON+Timer+COMMA)+Optional("Z"+COLON+Timer+COMMA)+digitMap)("digitMapValue")
    digitMapDescriptor=Group(DigitMapToken+EQUAL+((LBRKT+digitMapValue+RBRKT)| (digitMapName+Optional(LBRKT+digitMapValue+RBRKT))))("digitMap")
    modemType=Group(V32bisToken|V22bisToken|V18Token|V22Token|V32Token|V34Token|V90Token|V91Token|SynchISDNToken|extensionParameter)("modemType")
    modemDescriptor=Group(ModemToken+((EQUAL+modemType)|(LSBRKT+delimitedList(modemType, COMMA)+RSBRKT))+Optional(LBRKT+delimitedList(propertyParm, COMMA)+RBRKT))("modem")
    observedEventParameter=(eventStream|eventOther)
    observedEvent=Group(Optional(TimeStamp+LWSP+COLON)+LWSP+pkgdName+Optional(LBRKT+delimitedList(observedEventParameter,COMMA)+RBRKT))("observedEvent")
    observedEventsDescriptor=Group(ObservedEventsToken+EQUAL+RequestID+LBRKT+SkipTo(RBRKT)+RBRKT)("observedEvents")
    eventDM=Group(DigitMapToken+EQUAL((digitMapName)|(LBRKT+digitMapValue+RBRKT)))("eventDM")
    embedSig=Group(EmbedToken+LBRKT+signalsDescriptor+RBRKT)("embedSig")
    secondEventParameter=Forward()
    secondRequestedEvent=Group(pkgdName+Optional(LBRKT+delimitedList(secondEventParameter, COMMA)+RBRKT))("secondRequestedEvent")
    embedFirst=Group(EventsToken+Optional(EQUAL+RequestID+LBRKT+delimitedList(secondRequestedEvent,COMMA)+RBRKT))("embedFirst")
    embedNoSig=Group(EmbedToken+LBRKT+embedFirst+RBRKT)("embedNoSig")
    embedWithSig=Group(EmbedToken+LBRKT+signalsDescriptor+Optional(COMMA+embedFirst)+RBRKT)("embedWithSig")
    notifyRegulated=Group(NotifyRegulatedToken+Optional(LBRKT+(embedWithSig|embedNoSig)+RBRKT))("notifyRegulated")
    notifyBehaviour=Group(NotifyImmediateToken|notifyRegulated|NeverNotifyToken)("notifyBehaviour")
    eventParameter=Group((embedWithSig|embedNoSig|KeepActiveToken|eventDM|eventStream|eventOther|notifyBehaviour|ResetEventsDescriptorToken))("eventParameter")
    secondEventParameter<<(embedSig|KeepActiveToken|eventDM|eventStream|eventOther|notifyBehaviour|ResetEventsDescriptorToken)
    requestedEvent=Group(pkgdName+Optional(LBRKT+delimitedList(eventParameter, COMMA)+RBRKT))("requestedEvent")
    eventsDescriptor=Group(EventsToken+Optional(EQUAL+RequestID+LBRKT+delimitedList(requestedEvent,COMMA)+RBRKT))("events")
    MuxType=Group((H221Token|H223Token|H226Token|V76Token|extensionParameter|Nx64kToken))("MuxType")
    serviceStates=Group(ServiceStatesToken+EQUAL+serviceStatesValue)("serviceStates")
    eventBufferControlValue=Group(("OFF"|LockStepToken))("eventBufferControlValue")
    eventBufferControl=Group(BufferToken+EQUAL+eventBufferControlValue)("eventBufferControl")
    terminationStateParm=Group((propertyParm|serviceStates|eventBufferControl))("terminationStateParm")
    terminationStateDescriptor=Group(TerminationStateToken+LBRKT+delimitedList(terminationStateParm, COMMA)+RBRKT)("terminationState")
    eventSpecParameter=Group((eventStream|eventOther))("eventSpecParameter")
    eventSpec=Group(pkgdName+Optional(LBRKT+delimitedList(eventSpecParameter,COMMA)+RBRKT))("eventSpec")
    eventBufferDescriptor=Group(EventBufferToken+Optional(LBRKT+delimitedList(eventSpec, COMMA)+RBRKT))("eventBuffer")
    remoteDescriptor=(RemoteToken+LBRKT+octetString+RBRKT)("Remote")
    localDescriptor=(LocalToken+LBRKT+octetString+RBRKT)("Local")
    streamMode=(ModeToken+EQUAL+streamModes)
    reservedGroupMode=Group(ReservedGroupToken+EQUAL+(Literal("ON")|"OFF"))("reservedGroupMode")
    reservedValueMode=Group(ReservedValueToken+EQUAL+(Literal("ON")|"OFF"))("reservedValueMode")
    localParm=(streamMode|propertyParm|reservedValueMode|reservedGroupMode)
    localControlDescriptor=Group(LocalControlToken+LBRKT+delimitedList(localParm,COMMA)+RBRKT)("LocalControl")
    localControlDescriptor=Group(LocalControlToken+LBRKT+octetString+RBRKT)("LocalControl")
    streamParm=(localDescriptor|remoteDescriptor|localControlDescriptor|statisticsDescriptor)
    streamDescriptor=Group(StreamToken+EQUAL+StreamID+LBRKT+delimitedList(streamParm, COMMA)+RBRKT)("Stream")
    mediaParm=streamParm|streamDescriptor|terminationStateDescriptor
    mediaDescriptor=Group(MediaToken+LBRKT+delimitedList(mediaParm, COMMA)+RBRKT)("Media")
    terminationIDList=Group(LBRKT+delimitedList(TerminationID, COMMA)+RBRKT)("terminationIDList")
    muxDescriptor=Group(MuxToken+EQUAL+MuxType+terminationIDList)("mux")
    termIDList=Group((TerminationID|LSBRKT+TerminationID+OneOrMore(COMMA+TerminationID)+RSBRKT))("termIDList")
    errorDescriptor=Group(ErrorToken+EQUAL+ErrorCode+LBRKT+Optional(quotedString)+RBRKT)("error")
    serviceChangeReply=Group(ServiceChangeToken+EQUAL+termIDList+Optional(LBRKT+(errorDescriptor|serviceChangeReplyDescriptor)+RBRKT))("serviceChangeReply")
    serviceChangeRequest=Group(ServiceChangeToken+EQUAL+termIDList+LBRKT+serviceChangeDescriptor+RBRKT)("serviceChangeRequest")
    notifyReply=Group(NotifyToken+EQUAL+termIDList+Optional(LBRKT+errorDescriptor+RBRKT))("notifyReply")
    notifyRequest=Group(NotifyToken+EQUAL+termIDList+LBRKT+(observedEventsDescriptor+Optional(COMMA+errorDescriptor))+RBRKT)("notifyRequest")
    auditDescriptor=Group(AuditToken+LBRKT+Optional(delimitedList(auditItem,COMMA))+RBRKT)("audit")
    auditReturnParameter=(mediaDescriptor|modemDescriptor|muxDescriptor|eventsDescriptor|signalsDescriptor|digitMapDescriptor|observedEventsDescriptor|eventBufferDescriptor|statisticsDescriptor|packagesDescriptor|errorDescriptor|auditReturnItem)
    contextTerminationAudit=Group(EQUAL+CtxToken+(terminationIDList|LBRKT+errorDescriptor+RBRKT))("contextTerminationAudit")
    terminationAudit=Group(delimitedList(auditReturnParameter,COMMA))("terminationAudit")
    auditOther=Group(EQUAL+termIDList+Optional(LBRKT+terminationAudit+RBRKT))("auditOther")
    auditReply=Group((AuditValueToken|AuditCapToken)+(contextTerminationAudit|auditOther))("auditReply")
    auditRequest=Group((AuditValueToken|AuditCapToken)+EQUAL+termIDList+LBRKT+auditDescriptor+RBRKT)("auditRequest")
    subtractRequest=Group(SubtractToken+EQUAL+termIDList+Optional(LBRKT+auditDescriptor+RBRKT))("subtractRequest")
    ammsReply=Group((AddToken|MoveToken|ModifyToken|SubtractToken)+EQUAL+termIDList+Optional(LBRKT+terminationAudit+RBRKT))("ammsReply")
    ammParameter=mediaDescriptor|modemDescriptor|muxDescriptor|eventsDescriptor|signalsDescriptor|digitMapDescriptor|eventBufferDescriptor|auditDescriptor|statisticsDescriptor
    ammRequestBody=EQUAL+termIDList+Optional(LBRKT+delimitedList(ammParameter, COMMA)+RBRKT)
    ammRequest=Group((MoveToken|ModifyToken)+ ammRequestBody )("ammRequest")
    addRequest=Group(AddToken + ammRequestBody)("Add")
    segmentNumber=UINT16("segmentNumber")
    segmentReply=Group(MessageSegmentToken+EQUAL+TransactionID+SLASH+segmentNumber+Optional(SLASH+SegmentationCompleteToken))("segmentReply")
    commandRequest=addRequest| ammRequest | subtractRequest|auditRequest|notifyRequest|serviceChangeRequest
    commandRequestList=delimitedList(Optional("O-")+Optional("W-")+commandRequest, COMMA)
    AndAUDITselectToken=Group(Literal("TODO"))("AndAUDITselectToken")
    OrAUDITselectToken=Group(Literal("TODO"))("OrAUDITselectToken")
    auditSelectLogic=Group(Optional(AndAUDITselectToken|OrAUDITselectToken))("auditSelectLogic")
    contextAuditSelector=Group(priority|emergencyValue|iepsValue|contextAttrDescriptor|auditSelectLogic)("contextAuditSelector")
    contextAuditProperties=Group((TopologyToken|EmergencyToken|PriorityToken|IEPSToken|pkgdName|contextAuditSelector))("contextAuditProperties")
    indAudcontextAttrDescriptor=Group(ContextAttrToken+LBRKT+delimitedList(contextAuditProperties, COMMA)+RBRKT)("indAudcontextAttr")
    contextAudit=Group(ContextAuditToken+LBRKT+(delimitedList(contextAuditProperties,COMMA))|indAudcontextAttrDescriptor+RBRKT)("contextAudit")
    contextProperty=Group((topologyDescriptor|priority|EmergencyToken|EmergencyOffToken|iepsValue|contextAttrDescriptor))("contextProperty")
    contextProperties=Group(delimitedList(contextProperty,COMMA))("contextProperties")
    commandReplys=(serviceChangeReply|auditReply|ammsReply|notifyReply)
    commandReplyList=Group(delimitedList(commandReplys,COMMA))("commandReplyList")
    commandReply=Group(((contextProperties+Optional(COMMA+commandReplyList))|commandReplyList))("commandReply")
    actionReply=Group(CtxToken+EQUAL+ContextID+Optional(LBRKT+(errorDescriptor|commandReply|(commandReply+COMMA+errorDescriptor))+RBRKT))("actionReply")
    actionReplyList=Group(delimitedList(actionReply,COMMA))("actionReplyList")
    transactionReply=Group(ReplyToken+EQUAL+TransactionID+Optional(SLASH+segmentNumber+Optional(SLASH+SegmentationCompleteToken))+LBRKT+Optional(ImmAckRequiredToken+COMMA)+(errorDescriptor|actionReplyList)+RBRKT)("TransactionReply")
    contextRequest=Group(((contextProperties+Optional(COMMA+contextAudit))|contextAudit))("contextRequest")
    actionRequest=Group(CtxToken+EQUAL+ContextID+LBRKT+((contextRequest+Optional(COMMA+commandRequestList))|commandRequestList)+RBRKT)("actionRequest")
    transactionRequest=Group(TransToken+EQUAL+TransactionID+LBRKT+delimitedList(actionRequest, COMMA)+RBRKT)("TransactionRequest")
    transactionAck=Group(TransactionID|(TransactionID+"-"+TransactionID))("TransactionAck")
    transactionResponseAck=Group(ResponseAckToken+LBRKT+transactionAck+ZeroOrMore(COMMA+transactionAck)+RBRKT)("transactionResponseAck")
    transactionPending=Group(PendingToken+EQUAL+TransactionID+LBRKT+RBRKT)("transactionPending")
    transactionList=OneOrMore(transactionRequest|transactionReply|transactionPending|transactionResponseAck|segmentReply)
    messageBody=(transactionList | errorDescriptor)
    Message=Group(MegacopToken+SLASH+Version+SkipTo(lineEnd)+messageBody)("Message")
    AuthData=Group("0x"+Word(HEXDIG,min=24,max=64))("AuthData")
    SequenceNum=Group("0x"+Word(HEXDIG,exact=8))("SequenceNum")
    SecurityParmIndex=Group("0x"+Word(HEXDIG,exact=8))("SecurityParmIndex")
    authenticationHeader=Group(AuthToken+EQUAL+SecurityParmIndex+COLON+SequenceNum+COLON+AuthData)("authenticationHeader")
    megacoMessage=Group(LWSP+Optional(authenticationHeader+SEP)+Message)("megacoMessage").streamline()

1 个答案:

答案 0 :(得分:1)

您是否尝试过腌制解析器?我之前使用复杂的解析器(对于Python本身)已经完成了这项工作,并且在启动时间方面得到了合理的改进。将pickle解析器保存到文件中,然后解析单个消息,从文件中取消解析器而不是显式构建它。如果简化是设置的一个昂贵的部分,那么一定要在酸洗之前调用parser.streamline() - 然后在实际解析消息时,可以跳过这个昂贵的简化步骤。

如果您没有定义低级基元,而是一遍又一遍地显式重新定义相同的表达式,那么构建解析器也可能效率低下。例如,重复使用Word(nums),而不是定义低级表达式integer = Word(nums),然后在需要整数的位置引用integer。这很难在解析器构建时产生影响(我只看到另一个人实际上有这个问题),但它可能会发生。

您是否正在使用函数来定义遵循模式的复杂子解析器?尝试记住该函数,以便使用相同参数的重复调用返回相同的解析表达式而不是新表达式。然后,这些重复的表达式只会简化一次。

您是否有一个复杂的低级表达式,用于许多具有不同结果名称的地方?这实际上将创建该表达式的副本。您可以显式调用streamline作为定义步骤的一部分,并且副本将提前简化。如果你有一个真实数字的表达式,我可以设想这种事情。事实上,如果你用一个正则表达式(Combine(Optional(oneOf('- +')) + Word(nums) + Optional('.' + Word(nums)) + Optional(oneOf('E e') + Optional(oneOf('- +')) + Word(nums))组装pyparsing位(类似Regex(r'[-+]?\d+(\.\d*)?([Ee][-+]?\d+)?'))来替换一个实数定义,我已经看到了显着的分析时间改进(AddToken = Suppress(Literal("Add")| "A") 将对应于前一个表达)。不需要过分关注这一点,pyparsing将解析器表达式的许多内部表示作为正则表达式。

嗯,这就是我所能提出的所有问题 - 对于更具体的建议,你需要将你的500线路贴在某个地方的贴纸上,这样我们就可以看到正在发生的事情的细节。

编辑: 你可以做的一件事是尝试替换所有的文字定义,如:

AddToken = oneOf("Add A").suppress()

digitLetter=Group(Word(DIGIT+"-"+DIGIT)|Word(digitMapLetter))("digitLetter")

这将大大减少解析器中元素的数量,因此可以简化或腌制的位数更少。

这看起来像是一个错误:

Word(WSP)

没有理由在Word定义中重复DIGIT字符串。

不要创建包含空格的单词,您很可能会阅读超出预期的内容。您对LWSP的使用看起来也不合适,因为您正在尝试自己的空白跳过。如果必须执行此操作,请将White(WSP)更改为COMMENT = ';' + restOfLine + lineEnd 。但实际上,你为什么要自己进行空白跳过?

将评论更改为:

{{1}}