NiFi:无法将外部库添加到Python的

时间:2017-07-12 15:46:02

标签: python apache-nifi

我想在我的NiFi jython ExecuteScript处理器中使用Python数据库客户端进行Impala。

我已经使用pip install impyla安装了所需的模块,并在ExecuteScript Processor的Module Directory字段中指定了它的路径。

之后我运行脚本并收到错误javax.script.ScriptException: ImportError: No module named _bitarray

使用pip show bitarray我找到了这个模块位置并将其添加到Module Directory字段(使用逗号作为分隔符)并重启NiFi以防万一。

但它没有帮助。而且我也遇到了同样的错误。

经过一番研究后,我发现bitarray实际上是一个C扩展。

_bitarray出现在bitarray/__init__.py的下一行:

from bitarray._bitarray import _bitarray, bitdiff, bits2bytes, _sysinfo

反过来_bitarray存储在我在模块目录中指定的同一目录中的_bitarray.so文件中。

有人可以帮我解决如何加载bitarray模块的问题吗?

完整的堆栈跟踪:

  

2017-07-12 15:00:57,627 ERROR [定时器驱动的过程线程-5]   o.a.nifi.processors.script.ExecuteScript   org.apache.nifi.processor.exception.ProcessException:   javax.script.ScriptException:ImportError:没有名为_bitarray的模块   在第1行   org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:214)   〜[nifi-scripting-processors-1.1.2.jar:1.1.2] at   org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1099)   [nifi-framework-core-1.1.2.jar:1.1.2] at   org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)   [nifi-framework-core-1.1.2.jar:1.1.2] at   org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)   [nifi-framework-core-1.1.2.jar:1.1.2] at   org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent $ 1.run(TimerDrivenSchedulingAgent.java:132)   [nifi-framework-core-1.1.2.jar:1.1.2] at   java.util.concurrent.Executors $ RunnableAdapter.call(Executors.java:511)   [na:1.8.0_131] at   java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)   [na:1.8.0_131] at   java.util.concurrent.ScheduledThreadPoolExecutor中的$ ScheduledFutureTask.access $ 301(ScheduledThreadPoolExecutor.java:180)   [na:1.8.0_131] at   java.util.concurrent.ScheduledThreadPoolExecutor中的$ ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)   [na:1.8.0_131] at   java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)   [na:1.8.0_131] at   java.util.concurrent.ThreadPoolExecutor中的$ Worker.run(ThreadPoolExecutor.java:617)   [na:1.8.0_131]在java.lang.Thread.run(Thread.java:748)   [na:1.8.0_131]引起:javax.script.ScriptException:ImportError:   没有名为_bitarray的模块位于第1行   org.python.jsr223.PyScriptEngine.scriptException(PyScriptEngine.java:202)   〜[jython-standalone-2.7.0.jar:na] at   org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:42)   〜[jython-standalone-2.7.0.jar:na] at   org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:31)   〜[jython-standalone-2.7.0.jar:na] at   javax.script.AbstractScriptEngine.eval(AbstractScriptEngine.java:264)   〜[na:1.8.0_131] at   org.apache.nifi.processors.script.impl.JythonScriptEngineConfigurator.eval(JythonScriptEngineConfigurator.java:59)   〜[nifi-scripting-processors-1.1.2.jar:1.1.2] at   org.apache.nifi.processors.script.ExecuteScript.onTrigger(ExecuteScript.java:204)   〜[nifi-scripting-processors-1.1.2.jar:1.1.2] ... 11个常见帧   省略:org.python.core.PyException:null at   org.python.core.Py.ImportError(Py.java:328)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_logic(imp.java:912)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_module_level(imp.java:978)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importName(imp.java:1062)   〜[jython-standalone-2.7.0.jar:na] at   。org.python.core.ImportFunction的呼叫(适用的内置的.java:1280)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyObject。的呼叫(PyObject.java:431)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core的内置 导入(适用的内置的.java:1232)。   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importFromAs(imp.java:1156)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importFrom(imp.java:1132)   〜[jython-standalone-2.7.0.jar:na] at   bitarray $ $ py.f 0(/usr/local/lib/python2.7/dist-packages/bitarray/的初始化的.py:131)   〜[na:na] at   orar.python.core.PyTableCode.call中的bitarray $ py.call_function(/usr/local/lib/python2.7/dist-packages/bitarray/ init .py)〜[na:na] (PyTableCode.java:167)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyCode.call(PyCode.java:18)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromCode(imp.java:436)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromPyClass(imp.java:236)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromPyClass(imp.java:205)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.loadFromSource(imp.java:651)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.find_module(imp.java:543)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_next(imp.java:840)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_module_level(imp.java:959)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importName(imp.java:1062)   〜[jython-standalone-2.7.0.jar:na] at   。org.python.core.ImportFunction的呼叫(适用的内置的.java:1280)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyObject。的呼叫(PyObject.java:431)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core的内置 导入(适用的内置的.java:1232)。   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importFromAs(imp.java:1156)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importFrom(imp.java:1132)   〜[jython-standalone-2.7.0.jar:na] at   impala.hiveserver2 $ py.f $ 0(/usr/local/lib/python2.7/dist-packages/impala/hiveserver2.py:1211)   〜[na:na] at   impala.hiveserver2 $ py.call_function(/usr/local/lib/python2.7/dist-packages/impala/hiveserver2.py)   〜[na:na] at org.python.core.PyTableCode.call(PyTableCode.java:167)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyCode.call(PyCode.java:18)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromCode(imp.java:436)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromPyClass(imp.java:236)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromPyClass(imp.java:205)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.loadFromSource(imp.java:651)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.find_module(imp.java:543)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyModule.impAttr(PyModule.java:106)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_next(imp.java:842)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_logic(imp.java:904)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_module_level(imp.java:978)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importName(imp.java:1062)   〜[jython-standalone-2.7.0.jar:na] at   。org.python.core.ImportFunction的呼叫(适用的内置的.java:1280)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyObject。的呼叫(PyObject.java:431)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core的内置 导入(适用的内置的.java:1232)。   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importOneAs(imp.java:1099)   〜[jython-standalone-2.7.0.jar:na] at   impala.dbapi $ py.f $ 0(/usr/local/lib/python2.7/dist-packages/impala/dbapi.py:193)   〜[na:na] at   impala.dbapi $ py.call_function(/usr/local/lib/python2.7/dist-packages/impala/dbapi.py)   〜[na:na] at org.python.core.PyTableCode.call(PyTableCode.java:167)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyCode.call(PyCode.java:18)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromCode(imp.java:436)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromPyClass(imp.java:236)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.createFromPyClass(imp.java:205)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.loadFromSource(imp.java:651)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.find_module(imp.java:543)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyModule.impAttr(PyModule.java:106)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_next(imp.java:842)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_logic(imp.java:904)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.import_module_level(imp.java:978)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importName(imp.java:1062)   〜[jython-standalone-2.7.0.jar:na] at   。org.python.core.ImportFunction的呼叫(适用的内置的.java:1280)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyObject。的呼叫(PyObject.java:431)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core的内置 导入(适用的内置的.java:1232)。   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importFromAs(imp.java:1156)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.imp.importFrom(imp.java:1132)   〜[jython-standalone-2.7.0.jar:na] at   org.python.pycode._pyx1450.f $ 0(:5)〜[na:na] at   org.python.pycode._pyx1450.call_function()〜[na:na] at   org.python.core.PyTableCode.call(PyTableCode.java:167)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.PyCode.call(PyCode.java:18)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core.Py.runCode(Py.java:1386)   〜[jython-standalone-2.7.0.jar:na] at   org.python.core的内置 .eval(的内置的.java:497)。   〜[jython-standalone-2.7.0.jar:na] at   org.python.core的内置 .eval(的内置的.java:501)。   〜[jython-standalone-2.7.0.jar:na] at   org.python.util.PythonInterpreter.eval(PythonInterpreter.java:259)   〜[jython-standalone-2.7.0.jar:na] at   org.python.jsr223.PyScriptEngine.eval(PyScriptEngine.java:40)   〜[Jython的独立-2.7.0.jar:NA]

1 个答案:

答案 0 :(得分:1)

Jython无法加载原生(CPython,例如)模块,请参阅this related SO answer了解更多详情。