Question

我尝试使用multiprocessing.Pool并行执行一些CypherQuery（）。

当我运行neo4j.CypherQuery（）非并行时，它工作正常。当我在multiprocessing.Pool上只运行1个neo4j.CypherQuery（）时，它运行正常。一旦我启动了2个或更多neo4j.CypherQuery（）进程，它就会失败，并显示以下错误消息。

from mulitprocessing import Pool
from py2neo import neo4j

pool = Pool(processes=4)

db = neo4j.GraphDatabaseService("http://localhost:7474/db/data/")

def cypher_query(db):
    try:
        # very simple cypher query
        query_string = "MATCH (n:Label) RETURN n.name, n"
        query = neo4j.CypherQuery(db, query_string)
        result = query.execute()
        return_dict = {}

        for r in result:
            return_dict[r[0]] = r[1]
        return return_dict

    except:
        # print stack trace
        print('%s' % (traceback.format_exc()))

result1 = pool.apply_async(cypher_query, [db])
result2 = pool.apply_async(cypher_query, [db])

# close pool and wait for all processes to finish
pool.close()
pool.join()   

# here I would collect results, something fails before
result1.get()
result2.get()

错误讯息：

Traceback (most recent call last):
  File "/path/to/my/script.py", line 237, in my_function
    query = neo4j.CypherQuery(db, query_string)
  File "build/bdist.linux-x86_64/egg/py2neo/neo4j.py", line 976, in __init__
    self._cypher = Resource(graph_db.__metadata__["cypher"])
  File "build/bdist.linux-x86_64/egg/py2neo/neo4j.py", line 320, in __metadata__
    self.refresh()
  File "build/bdist.linux-x86_64/egg/py2neo/neo4j.py", line 342, in refresh
    self._metadata = ResourceMetadata(self._get().content)
  File "build/bdist.linux-x86_64/egg/py2neo/packages/httpstream/http.py", line 532, in content
    elif self.is_text:
  File "build/bdist.linux-x86_64/egg/py2neo/packages/httpstream/http.py", line 513, in is_text
    return self.content_type.partition("/")[0] == "text"
AttributeError: 'NoneType' object has no attribute 'partition'

我没有收到错误消息。我尝试了不同的Cypher查询以及execute（）和stream（），但总是失败。所有查询都运行良好的非并行。显然，我遗漏了一些破坏我的功能并行化的东西，但我不知道如何解决它。

Answer 1

我知道py2neo / httpstream不适用于多处理，GitHub中存在一个问题：

https://github.com/nigelsmall/httpstream/issues/3

但是，由于我自己并不需要多处理，并且对此模块知之甚少，因此在解决此问题方面没有取得任何进展。我很乐意让贡献者研究这个并提供补丁，但到目前为止还没有人自愿这样做。

Answer 2

在版本1.6.1中的数据库连接字符串中使用http而不是https时也会出现此错误

py2neo：neo4j.CypherQuery（）在multiprocessing.Pool中失败

2 个答案: