Question

我有2个日志文件。我需要根据时间戳合并这两个文件。我正在努力寻找正确的算法来做同样的事情。

我有文件＆＃39; File1＆＃39;内容：

2016-07-18 09:58:19,243 : INFO: My Test File1 - 1
2016-07-18 09:58:19,244 : INFO: My Test File1 - 2
2016-07-18 09:58:19,255 : INFO: My Test File1 - 3
2016-07-18 09:58:19,255 : INFO: My Test File1 - 4
2016-07-18 09:58:19,258 : INFO: My Test File1 - 5

＆＃39; File2＆＃39;内容：

2016-07-18 09:57:09,674 : INFO: My Test File2 - 1
2016-07-18 09:57:09,674 : INFO: My Test File2 - 2
2016-07-18 09:57:09,679 : INFO: My Test File2 - 3
2016-07-18 09:57:09,679 : INFO: My Test File2 - 4
2016-07-18 09:57:09,680 : INFO: My Test File2 - 5
2016-07-18 09:58:49,685 : INFO: My Test File2 - 6
2016-07-18 09:58:49,686 : INFO: My Test File2 - 7
2016-07-18 09:58:49,686 : INFO: My Test File2 - 8

合并后

预期输出文件：

2016-07-18 09:57:09,674 : INFO: My Test File2 - 1
2016-07-18 09:57:09,674 : INFO: My Test File2 - 2
2016-07-18 09:57:09,679 : INFO: My Test File2 - 3
2016-07-18 09:57:09,679 : INFO: My Test File2 - 4
2016-07-18 09:57:09,680 : INFO: My Test File2 - 5
2016-07-18 09:58:19,243 : INFO: My Test File1 - 1
2016-07-18 09:58:19,244 : INFO: My Test File1 - 2
2016-07-18 09:58:19,255 : INFO: My Test File1 - 3
2016-07-18 09:58:19,255 : INFO: My Test File1 - 4
2016-07-18 09:58:19,258 : INFO: My Test File1 - 5
2016-07-18 09:58:49,685 : INFO: My Test File2 - 6
2016-07-18 09:58:49,686 : INFO: My Test File2 - 7
2016-07-18 09:58:49,686 : INFO: My Test File2 - 8

以下是我编写的用于合并2文件的代码。

private static final String DATE_FORMAT_REGEX = "(\\d{4})-(\\d{2})-(\\d{2}) (\\d{2}):(\\d{2}):(\\d{2}).*";
private static final Pattern DATE_FORMAT_PATTERN = Pattern.compile(DATE_FORMAT_REGEX);
private static final String COLON = " : ";
// 2016-07-18 09:57:09,674
private static final String DATE_PATTERN = "yyyy-MM-dd HH:mm:ss,SSS";
File file1 = new File("C:\\Users\\Admin\\Downloads\\log.log");
File file2 = new File("C:\\Users\\Admin\\Downloads\\log2.log");

LineIterator it = FileUtils.lineIterator(file1,"UTF-8");
LineIterator it1 = FileUtils.lineIterator(file2,"UTF-8");
PrintWriter writer = new PrintWriter("C:\\logMerge.txt", "UTF-8");
SimpleDateFormat sdf = new SimpleDateFormat(DATE_PATTERN);
      try {
            while (it.hasNext()) {
            String line = it.nextLine();
                    String date = line.split(COLON)[0].trim();
                    Matcher matcher = DATE_FORMAT_PATTERN.matcher(date);
                    if (matcher.matches()) {
                        System.out.println(line);
                        while (it1.hasNext()) {
                            String line1 = it1.nextLine();
                            String date1 = line1.split(COLON)[0].trim();
                            Matcher matcher1 = DATE_FORMAT_PATTERN.matcher(date1);
                            if (matcher1.matches()) {
                                Date convertedDate = sdf.parse(date);
                                Date convertedDate1 = sdf.parse(date1);
                                if (convertedDate.before(convertedDate1)) {

                                    writer.println(line);
                                    break;
                                }
                                else if (convertedDate1.before(convertedDate)) {
                                    writer.println(line1);
                                }
                                else {
                                    writer.println(line1);
                                }
                            }
                            else {
                                writer.println(line1);
                            }
                        }   
                    }
                    else {
                        writer.println(line);
                    }
                }
            }
            catch (ParseException e) {
                // TODO Auto-generated catch block
                e.printStackTrace();
            }
            finally {
                it.close();
            }

但是使用上面的代码我无法合并两个记录。任何帮助表示赞赏。

编辑：

实际结果

2016-07-18 09:57:09,674 : INFO: My Test File2 - 1
2016-07-18 09:57:09,674 : INFO: My Test File2 - 2
2016-07-18 09:57:09,679 : INFO: My Test File2 - 3
2016-07-18 09:57:09,679 : INFO: My Test File2 - 4
2016-07-18 09:57:09,680 : INFO: My Test File2 - 5
2016-07-18 09:58:49,685 : INFO: My Test File2 - 6
2016-07-18 09:58:49,686 : INFO: My Test File2 - 7
2016-07-18 09:58:49,686 : INFO: My Test File2 - 8

Answer 1

我甚至不打算解析日期。使用sort进行简单的字母数字排序可以提供正确的结果，因为日志编写者可以很好地编写符合字母格式的年 - 月 - 日格式（否则需要进行日期解析）

MSYS / linux（使用GNU排序）

sort File1 File2 > Sorted.txt

Windows CMD-line（使用Windows排序）：

(type File1 & type File2) | sort > Sorted.txt

除非你绝对想要为此烦恼，但由于你正在编写一个主程序，我怀疑它等同于cmdline解决方案。

Answer 2

恕我直言，基本算法如下：

获取FileList。
创建关于的BufferedReader列表文件。
阅读每行的第1行并将其存储在另一个列表中。
存储每个文件第1行的日期。
当BufferedReader的大小不为零时，（如果5为真，则重复6-12的过程）
从dateList
获取上一步中获得的索引（lineToWrite），然后编写 lineToWrite。
获取带索引的缓冲读卡器。
如果BR不为空，则读取该行。
如果line不等于null，则从索引中删除lineList并将该行添加到index。如果line为null，则从中删除最小索引行 lineList，dateList，BufferedReader列表。 Do BufferedReader--。
如果线长大于23（yyyy-MM-dd HH：mm：ss，SSS）则采取该行的前23个字符串。检查日期是否与日期匹配 pattern，如果匹配则将日期添加到dateList
重复步骤5的过程

以下是组合日志文件的代码。

    public static void main(String[] args) throws Exception {
    final String DATE_FORMAT_REGEX = "(\\d{4})-(\\d{2})-(\\d{2}) (\\d{2}):(\\d{2}):(\\d{2}).*";
    final Pattern DATE_FORMAT_PATTERN = Pattern.compile(DATE_FORMAT_REGEX);
    final String DATE_PATTERN = "yyyy-MM-dd HH:mm:ss,SSS";
    SimpleDateFormat sdf = new SimpleDateFormat(DATE_PATTERN);
    String file1 = "C:\\templog\\myLog1.log";
    String file2 = "C:\\templog\\myLog2.log";
    PrintWriter writer = new PrintWriter("C:\\templog\\expectedLog.txt", "UTF-8");
    // 1) Get the file list
    List<String> fileFile = Arrays.asList(file1, file2);
    ArrayList<BufferedReader> readerIndex = new ArrayList<BufferedReader>();
    ArrayList<String> lineList = new ArrayList<String>();
    ArrayList<Date> dateList = new ArrayList<Date>();
    for (String file : fileFile) {
        // 2) Create BufferedReader list with respect to the file.
        BufferedReader br1 = new BufferedReader(new FileReader(file));
        readerIndex.add(br1);
        // 3) Read the 1st line of each line and store it in another list.
        String line = br1.readLine();
        lineList.add(line);
        // 4) Store the date from the 1st line of each file.
        String date = line.substring(0, 23);
        Date convertedDate = sdf.parse(date);
        dateList.add(convertedDate);
    }
    int index = readerIndex.size();
    // 5) While BufferedReader's size is not zero then,
    while (index > 0) {
        // 6) Take the index of minimum date from dateList
        int indexMin = minDateIndex(dateList);
        // 7) Get the Line with the index you got in the previous step (lineToWrite).
        String lineToWrite = lineList.get(indexMin);
        writer.println(lineToWrite);
        // 8) Get the buffered reader with the index.
        BufferedReader br1 = readerIndex.get(indexMin);
        if (br1 != null) {
            //  9) If the BR is not null then read the line.
            String line = br1.readLine();
            if (line != null) {
                // 10)If line is not equal to null then remove the lineList from the index and add the line to index.
                lineList.remove(indexMin);
                lineList.add(indexMin, line);
                if (line.length() > 23) {
                    // 11)If the line length is greater than 23 (yyyy-MM-dd HH:mm:ss,SSS) then take the first 23 String from the line.
                    String date = line.substring(0, 23);
                    Matcher matcher = DATE_FORMAT_PATTERN.matcher(date);
                    if (matcher.matches()) {
                        // 12)Check if the date matches to the pattern, if matches then add the date to dateList
                        Date convertedDate = sdf.parse(date);
                        dateList.remove(indexMin);
                        dateList.add(indexMin, convertedDate);
                    }
                }
            } else {
                //If line is null then remove the min indexed line from lineList,dateList,BufferedReader list. Do BufferedReader--.
                lineList.remove(indexMin);
                dateList.remove(indexMin);
                readerIndex.remove(indexMin);
                br1.close();
                index--;
            }
        }
    }
    writer.close();
}

private static int minDateIndex(ArrayList<Date> dateList) {
    // return index of min date
    Date minDate = dateList.get(0);
    int minIndex = 0;
    for (int i = 1; i < dateList.size(); i++) {
        Date currentDate = dateList.get(i);
        if (minDate != null) {
            if (currentDate != null) {
                if (minDate.after(currentDate)) {
                    // We have a new min
                    minDate = dateList.get(i);
                    minIndex = i;
                } else {
                    // we keep current min
                }
            } else {
                // we keep current min
            }
        } else {
            minDate = currentDate;
            minIndex = i;
        }
    }
    return minIndex;
}

测试：

myLog1.log

2016-08-24 16:32:13,888[main] INFO  org.hibernate.cfg.Environment - HHH000206: hibernate.properties not found
2016-08-24 16:40:13,888 [main] INFO  o.h.tool.hbm2ddl.SchemaUpdate - HHH000396: Updating schema

myLog2.log

2016-08-24 16:33:13,888 : SEVERE : http-bio-8983-exec-3 : StandardWrapperValve.invoke :    Servlet.service() for servlet [Faces Servlet] in context with path [/TestApp] threw exception [Cannot create a session after the response has been committed] with root cause
java.lang.IllegalStateException: Cannot create a session after the response has been committed
    at org.apache.catalina.connector.Request.doGetSession(Request.java:2882)
    at org.apache.catalina.connector.Request.getSession(Request.java:2316)
    at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:898)
    at com.sun.faces.context.ExternalContextImpl.getSession(ExternalContextImpl.java:155)
    at com.sun.faces.mgbean.BeanManager$ScopeManager$SessionScopeHandler.handle(BeanManager.java:575)
    at com.sun.faces.mgbean.BeanManager$ScopeManager.pushToScope(BeanManager.java:458)
    at com.sun.faces.mgbean.BeanManager.createAndPush(BeanManager.java:410)
    at com.sun.faces.mgbean.BeanManager.create(BeanManager.java:269)
    at com.sun.faces.el.ManagedBeanELResolver.resolveBean(ManagedBeanELResolver.java:244)
    at com.sun.faces.el.ManagedBeanELResolver.getValue(ManagedBeanELResolver.java:116)
    at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
    at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
    at org.apache.el.parser.AstIdentifier.getValue(AstIdentifier.java:72)
    at org.apache.el.parser.AstValue.getTarget(AstValue.java:94)
    at org.apache.el.parser.AstValue.setValue(AstValue.java:210)
    at org.apache.el.ValueExpressionImpl.setValue(ValueExpressionImpl.java:253)
    at org.apache.jasper.el.JspValueExpression.setValue(JspValueExpression.java:89)
    at javax.faces.component.UIComponent.processEvent(UIComponent.java:2185)
    at com.sun.faces.lifecycle.RestoreViewPhase$1.visit(RestoreViewPhase.java:276)
    at com.sun.faces.component.visit.FullVisitContext.invokeVisitCallback(FullVisitContext.java:151)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1446)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIForm.visitTree(UIForm.java:333)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at com.sun.faces.component.visit.VisitUtils.doFullNonIteratingVisit(VisitUtils.java:75)
    at com.sun.faces.lifecycle.RestoreViewPhase.deliverPostRestoreStateEvent(RestoreViewPhase.java:271)
    at com.sun.faces.lifecycle.RestoreViewPhase.execute(RestoreViewPhase.java:251)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:111)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:312)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:349)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.ow2.jonas.web.tomcat7.CheckOpenResourcesValve.invoke(CheckOpenResourcesValve.java:69)
    at org.ow2.jonas.web.tomcat7.tx.TransactionValve.invoke(TransactionValve.java:89)
    at org.ow2.jonas.web.tomcat7.ResetAuthenticationValve.invoke(ResetAuthenticationValve.java:95)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.ow2.jonas.web.tomcat7.versioning.VersioningValve.invoke(VersioningValve.java:105)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)

在expectedLog.txt中运行程序

2016-08-24 16:32:13,888[main] INFO  org.hibernate.cfg.Environment - HHH000206: hibernate.properties not found
2016-08-24 16:33:13,888 : SEVERE : http-bio-8983-exec-3 : StandardWrapperValve.invoke :    Servlet.service() for servlet [Faces Servlet] in context with path [/TestApp] threw exception [Cannot create a session after the response has been committed] with root cause
java.lang.IllegalStateException: Cannot create a session after the response has been committed
    at org.apache.catalina.connector.Request.doGetSession(Request.java:2882)
    at org.apache.catalina.connector.Request.getSession(Request.java:2316)
    at org.apache.catalina.connector.RequestFacade.getSession(RequestFacade.java:898)
    at com.sun.faces.context.ExternalContextImpl.getSession(ExternalContextImpl.java:155)
    at com.sun.faces.mgbean.BeanManager$ScopeManager$SessionScopeHandler.handle(BeanManager.java:575)
    at com.sun.faces.mgbean.BeanManager$ScopeManager.pushToScope(BeanManager.java:458)
    at com.sun.faces.mgbean.BeanManager.createAndPush(BeanManager.java:410)
    at com.sun.faces.mgbean.BeanManager.create(BeanManager.java:269)
    at com.sun.faces.el.ManagedBeanELResolver.resolveBean(ManagedBeanELResolver.java:244)
    at com.sun.faces.el.ManagedBeanELResolver.getValue(ManagedBeanELResolver.java:116)
    at com.sun.faces.el.DemuxCompositeELResolver._getValue(DemuxCompositeELResolver.java:176)
    at com.sun.faces.el.DemuxCompositeELResolver.getValue(DemuxCompositeELResolver.java:203)
    at org.apache.el.parser.AstIdentifier.getValue(AstIdentifier.java:72)
    at org.apache.el.parser.AstValue.getTarget(AstValue.java:94)
    at org.apache.el.parser.AstValue.setValue(AstValue.java:210)
    at org.apache.el.ValueExpressionImpl.setValue(ValueExpressionImpl.java:253)
    at org.apache.jasper.el.JspValueExpression.setValue(JspValueExpression.java:89)
    at javax.faces.component.UIComponent.processEvent(UIComponent.java:2185)
    at com.sun.faces.lifecycle.RestoreViewPhase$1.visit(RestoreViewPhase.java:276)
    at com.sun.faces.component.visit.FullVisitContext.invokeVisitCallback(FullVisitContext.java:151)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1446)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at javax.faces.component.UIForm.visitTree(UIForm.java:333)
    at javax.faces.component.UIComponent.visitTree(UIComponent.java:1457)
    at com.sun.faces.component.visit.VisitUtils.doFullNonIteratingVisit(VisitUtils.java:75)
    at com.sun.faces.lifecycle.RestoreViewPhase.deliverPostRestoreStateEvent(RestoreViewPhase.java:271)
    at com.sun.faces.lifecycle.RestoreViewPhase.execute(RestoreViewPhase.java:251)
    at com.sun.faces.lifecycle.Phase.doPhase(Phase.java:101)
    at com.sun.faces.lifecycle.RestoreViewPhase.doPhase(RestoreViewPhase.java:111)
    at com.sun.faces.lifecycle.LifecycleImpl.execute(LifecycleImpl.java:118)
    at javax.faces.webapp.FacesServlet.service(FacesServlet.java:312)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:305)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.myfaces.webapp.filter.ExtensionsFilter.doFilter(ExtensionsFilter.java:349)
    at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:243)
    at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:210)
    at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:222)
    at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:123)
    at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:472)
    at org.ow2.jonas.web.tomcat7.CheckOpenResourcesValve.invoke(CheckOpenResourcesValve.java:69)
    at org.ow2.jonas.web.tomcat7.tx.TransactionValve.invoke(TransactionValve.java:89)
    at org.ow2.jonas.web.tomcat7.ResetAuthenticationValve.invoke(ResetAuthenticationValve.java:95)
    at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:171)
    at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:99)
    at org.apache.catalina.valves.AccessLogValve.invoke(AccessLogValve.java:947)
    at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:118)
    at org.ow2.jonas.web.tomcat7.versioning.VersioningValve.invoke(VersioningValve.java:105)
    at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:408)
    at org.apache.coyote.http11.AbstractHttp11Processor.process(AbstractHttp11Processor.java:1009)
    at org.apache.coyote.AbstractProtocol$AbstractConnectionHandler.process(AbstractProtocol.java:589)
    at org.apache.tomcat.util.net.JIoEndpoint$SocketProcessor.run(JIoEndpoint.java:310)
    at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:895)
    at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:918)
    at java.lang.Thread.run(Thread.java:662)
2016-08-24 16:40:13,888 [main] INFO  o.h.tool.hbm2ddl.SchemaUpdate - HHH000396: Updating schema

希望它有所帮助！

Answer 3

我没有仔细查看代码，但是如果您可以先对文件进行排序然后以锯齿形方式阅读它们，那么您就可以实现所需的目标。

zig-zag我的意思是 -

从两个文件中读取第一行。拆分它们并找到时间戳，比如T1和T2。
如果T1 <＆lt; T2然后从第一个文件（在输出文件中）写入T1行
现在从第一个文件中读取下一行。假设这次是T3。
检查T3＆lt; T2然后在输出文件中写入T3行并仅从第一个文件读取下一行。如果T2 <＆lt; T3然后在输出文件中写入T2行并从第二个文件读取下一行。

玩得开心!!!

Answer 4

我认为有一个最好的方法可以解决这个问题：

将每一行解析为对象＆＃39; MyLog＆＃39; （实现Comparator）具有属性：date，info ...

实施方法＆＃39;比较＆＃39;日期之间。

    @Override
    public int compare(Date o1, Date o2) {
        return o1.compareTo(o2);
    }

将此对象添加到ArrayList
使用方法排序。

也许这个链接就是您的答案：https://stackoverflow.com/a/5927408/2269677

Java：按日期合并多个日志文件

4 个答案: