如何从日志文件中提取Java堆栈跟踪的时间戳?

时间:2019-03-21 17:32:41

标签: python python-2.x

这是来自WebSphere日志文件的示例-

[1/31/19 13:55:55:567 CET] 0000099c ThreadPoolTas I org.springframework.scheduling.concurrent.ExecutorConfigurationSupport shutdown Shutting down ExecutorService 'sr_taskExecutor'
[1/31/19 13:55:55:568 CET] 0000099c appDispatcher E org.springframework.web.servlet.FrameworkServlet initServletBean Context initialization failed
                                 org.springframework.beans.factory.UnsatisfiedDependencyException: Error creating bean with name 'messageSource' defined in class path resource [context/main/portal-main-context.xml]: Unsatisfied dependency expressed through constructor argument with index 0 of type [no.myapp.portal.core.internal.PortalUserRequestData]: : Cannot find class [no.myapp.api.common.ProxiedClientServiceFactory] for bean with name 'myappapiFactory' defined in ServletContext resource [/WEB-INF/app-context.xml]; nested exception is java.lang.ClassNotFoundException: no.myapp.api.common.ProxiedClientServiceFactory; nested exception is org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [no.myapp.api.common.ProxiedClientServiceFactory] for bean with name 'myappapiFactory' defined in ServletContext resource [/WEB-INF/app-context.xml]; nested exception is java.lang.ClassNotFoundException: no.myapp.api.common.ProxiedClientServiceFactory
Related cause: org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [no.myapp.api.common.ProxiedClientServiceFactory] for bean with name 'myappapiFactory' defined in ServletContext resource [/WEB-INF/app-context.xml]; nested exception is java.lang.ClassNotFoundException: no.myapp.api.common.ProxiedClientServiceFactory
Related cause: org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [no.myapp.api.common.ProxiedClientServiceFactory] for bean with name 'myappapiFactory' defined in ServletContext resource [/WEB-INF/app-context.xml]; nested exception is java.lang.ClassNotFoundException: no.myapp.api.common.ProxiedClientServiceFactory
Related cause: org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [no.myapp.api.common.ProxiedClientServiceFactory] for bean with name 'myappapiFactory' defined in ServletContext resource [/WEB-INF/app-context.xml]; nested exception is java.lang.ClassNotFoundException: no.myapp.api.common.ProxiedClientServiceFactory
Related cause: org.springframework.beans.factory.BeanCreationException: Error creating bean with name 'dataSource_db811': Invocation of init method failed; nested exception is javax.naming.NameNotFoundException: Context: Cell-01, name: ds_appng_811: First component in name ds_appng_811 not found. [Root exception is org.omg.CosNaming.NamingContextPackage.NotFound: IDL:omg.org/CosNaming/NamingContext/NotFound:1.0]
    at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:752)
    at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:193)
    at com.ibm.ws.runtime.component.CompositionUnitMgrImpl$CUInitializer.run(CompositionUnitMgrImpl.java:985)
    at com.ibm.wsspi.runtime.component.WsComponentImpl$_AsynchInitializer.run(WsComponentImpl.java:524)
    at com.ibm.ws.util.ThreadPool$Worker.run(ThreadPool.java:1892)
Caused by: org.springframework.beans.factory.CannotLoadBeanClassException: Cannot find class [no.myapp.api.common.ProxiedClientServiceFactory] for bean with name 'myappapiFactory' defined in ServletContext resource [/WEB-INF/app-context.xml]; nested exception is java.lang.ClassNotFoundException: no.myapp.api.common.ProxiedClientServiceFactory
    at org.springframework.beans.factory.support.AbstractBeanFactory.resolveBeanClass(AbstractBeanFactory.java:1281)
    at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.predictBeanType(AbstractAutowireCapableBeanFactory.java:575)
    at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:745)
    ... 47 more
Caused by: java.lang.ClassNotFoundException: no.myapp.api.common.ProxiedClientServiceFactory
    at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
    at com.ibm.ws.bootstrap.ExtClassLoader.findClass(ExtClassLoader.java:243)
    at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
    ********Skip Trace************
[1/31/19 13:55:57:004 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain HttpQueueInboundDefaultSecure.
[1/31/19 13:55:57:006 CET] 00000001 TCPChannel    I   TCPC0001I: TCP Channel TCP_3 is listening on host *  (IPv6) port 9048.
[1/31/19 13:55:57:008 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain WCInboundAdminSecure.
[1/31/19 13:55:57:010 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain WCInboundDefaultSecure.
[1/31/19 13:55:57:011 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain SOAPAcceptorChain1.
[1/31/19 13:55:57:012 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain SOAPAcceptorChain2.
[1/31/19 13:55:57:012 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain SOAPAcceptorChain3.
[1/31/19 13:55:57:013 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain SOAPAcceptorChain4.
[1/31/19 13:55:57:014 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain SOAPAcceptorChain5.
[1/31/19 13:55:57:014 CET] 00000001 WSChannelFram A   CHFW0019I: The Transport Channel Service has started chain SOAPAcceptorChain6.

在这里,我想获取org.omg.CosNaming.NamingContextPackage.NotFound事件的时间戳,该时间戳位于(1/31/19 13:55:55:568 CET)上方几行。

如果目标字符串出现在单行中,那么到目前为止,下面的代码仍有效-

code="org.omg.CosNaming.NamingContextPackage.NotFound"
with open('Server01/SystemOut.log',"r") as data:
    for i in data:
       if code in i.strip():
          val =  i[i.find("[")+1:i.find("]")].strip()
          print i ##Result stack but single line
          print val  ##Result only timestamp

预期结果: 针对特定异常字符串的时间戳列表

org.omg.CosNaming.NamingContextPackage.NotFound - ['1/31/19 13:55:55:568 CET']

4 个答案:

答案 0 :(得分:1)

如果可能的话,我建议使用正则表达式跟踪时间戳:

import re


# before the loop
timestamp_pattern = re.compile("\[([\d/\s:]+\s+CET)\]") 
timestamp_str  =""
code="org.omg.CosNaming.NamingContextPackage.NotFound"


with open() ... as lines:
  for line in lines:

     # line could be like this for example
     line = "[1/31/19 13:55:55:568 CET] blablabla" 
     tmatch = timestamp_pattern.search(line) 
     # if no match timestamp_str would not be changed 
     if tmatch:
        timestamp_str = tmatch.group(0)
     if code in line:
        print(time_stamp_str)

答案 1 :(得分:1)

以下是带有注释的日志过滤错误示例:

def contains_timestamp(s):
    return s.startswith('[') and 'CET' in s

exception = 'org.omg.CosNaming.NamingContextPackage.NotFound'
errors = []
with open('log.txt', "r") as fl:
    data = fl.readlines()
i = 0  # line number
while i < len(data):
    s = data[i]
    i += 1
    if not contains_timestamp(s):  # we are looking for timestamp
        continue
    timestamp =  s[s.find("[") + 1:s.find("]")].strip()
    while i < len(data) and not contains_timestamp(data[i]) and exception not in data[i]:  # we are looking for line with exception
        i += 1
    if i == len(data):  # EOF reached
        break
    s = data[i]
    if contains_timestamp(s):  # timestamp means that we found new log entry
        continue
    errors.append(timestamp)
    i += 1

print errors

答案 2 :(得分:1)

使用正则表达式检查行是否以时间戳开头,并在找到新时间戳时更新时间戳。

如果在以下行中找到代码,则继续遍历文件并汇总当前时间戳:

import re
pat = re.compile('^\[\d{1,2}/\d{1,2}/\d{2} \d{1,2}:\d{1,2}:\d{1,2}:\d{3} \w{3}\]')
tstamp = None # initally current tstamp is None
tstamps = []  # initially current tstamps is empty list
code = 'org.omg.CosNaming.NamingContextPackage.NotFound'

with open('Server01/SystemOut.log') as lines:
    for line in lines:
        if re.match(pat, line):
            tstamp = re.findall(pat, line)[0]
        if code in line:
            # append the tstamp excluding terminal square brackets
            tstamps.append(tstamp[1:-1]) 

print('{c} - {t}'.format(c=code, t=tstamps))

随样本数据一起打印

org.omg.CosNaming.NamingContextPackage.NotFound - ['1/31/19 13:55:55:568 CET']

答案 3 :(得分:1)

您可以结合使用re模块中的正则表达式和generator函数。生成器函数将紧随其后的所有行(其中带有时间戳)分组在一起。

import re

code = "org.omg.CosNaming.NamingContextPackage.NotFound"
timestamp_pattern = re.compile("\[([\d/\s:]+\s+CET)\]")
#log_filename = 'Server01/SystemOut.log'
log_filename = 'Server01_SystemOut.log'

def read_blocks(log_filename):
    """ Group lines following one with a timestamp together with it. """
    with open(log_filename, "r") as data:
        block = []
        for line in data:
            if not timestamp_pattern.search(line):  # Continue current block?
                block.append(line)
            else:
                if block:  #  Previous block?
                    yield(''.join(block))
                block = [line]  # Start new block.

for block in read_blocks(log_filename):
    if code in block:
        timestamp = timestamp_pattern.search(block).group()
        print('timestamp: {}'.format(timestamp))
        print(block)