Question

我遇到了NIFI服务拒绝启动的问题。我监控了nifi-app.log，发现它是OutOfMemory错误。我增加了bootstap.conf文件中的内存。但错误仍然存在。

附件是nifi-app.log

    2017-12-19 17:08:37,133 ERROR [ActiveMQ InactivityMonitor ReadCheckTimer] org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[ActiveMQ InactivityMonitor ReadCheckTimer,5,main]: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:37,134 ERROR [Provenance Maintenance Thread-2] org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[Provenance Maintenance Thread-2,5,main]: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:37,134 ERROR [logback-2] org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[logback-2,5,main]: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:37,133 ERROR [FileSystemRepository Workers Thread-2] org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[FileSystemRepository Workers Thread-2,5,main]: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:37,135 ERROR [pool-15-thread-1] org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[pool-15-thread-1,5,main]: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:37,133 ERROR [Flow Service Tasks Thread-1] org.apache.nifi.NiFi An Unknown Error Occurred in Thread Thread[Flow Service Tasks Thread-1,5,main]: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:39,496 ERROR [Provenance Maintenance Thread-2] org.apache.nifi.NiFi 
java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:39,496 ERROR [logback-2] org.apache.nifi.NiFi 
java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:39,496 ERROR [FileSystemRepository Workers Thread-2] org.apache.nifi.NiFi 
java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:38,482 ERROR [Flow Service Tasks Thread-1] org.apache.nifi.NiFi 
java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:38,482 ERROR [Event-Driven Process Thread-1] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:38,483 ERROR [FileSystemRepository Workers Thread-4] org.apache.nifi.engine.FlowEngine A flow controller task execution stopped abnormally
java.util.concurrent.ExecutionException: java.lang.OutOfMemoryError: Java heap space
        at java.util.concurrent.FutureTask.report(FutureTask.java:122)
        at java.util.concurrent.FutureTask.get(FutureTask.java:192)
        at org.apache.nifi.engine.FlowEngine.afterExecute(FlowEngine.java:100)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1150)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:38,482 ERROR [pool-15-thread-1] org.apache.nifi.NiFi 
java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:39,497 ERROR [ActiveMQ InactivityMonitor ReadCheckTimer] org.apache.nifi.NiFi 
java.lang.OutOfMemoryError: Java heap space
2017-12-19 17:08:40,509 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@5351d0ef checkpointed with 2358 Records and 0 Swap Files in 114969 milliseconds (Stop-the-world time = 2361 milliseconds, Clear Edit Logs time = 936 millis), max Transaction ID 7622
2017-12-19 17:10:41,433 INFO [Write-Ahead Local State Provider Maintenance] org.wali.MinimalLockingWriteAheadLog org.wali.MinimalLockingWriteAheadLog@5351d0ef checkpointed with 2358 Records and 0 Swap Files in 922 milliseconds (Stop-the-world time = 2 milliseconds, Clear Edit Logs time = 2 millis), max Transaction ID 7622

我还试图删除流文件和内容存储库，因为它担心它可能已被破坏。但是，仍然没有变化。

我能理解的是它是一个线程错误，但我无法找到它的解决方法。

Answer 1

尝试关闭，删除repos，并在nifi.properties中设置nifi.flowcontroller.autoResumeState = false，然后重新开始。

这将确保当NiFi启动时，整个流程将停止，因此您的流程无法引入更多数据并导致内存不足错误，然后您可以启动部分流程以查看问题所在是

OOM的典型原因是获取大量流文件并将其拆分为数百万个小流量文件，例如具有1000万行的CSV进入SplitText，行数为1。

另一个原因可能是使用处理器将流文件的整个内容读入内存并使用多GB流文件命中该处理器，或使用ExtractText将大内容提取到属性中。

NIFI：内存错误，尽管堆大小增加

1 个答案: