尝试迁移项目listed here 哪个使用多处理,我收到以下错误:
/usr/lib/python3/dist-packages/apport/report.py:13: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import fnmatch, glob, traceback, errno, sys, atexit, locale, imp
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "PubMedParser.py", line 593, in _start_parser
s = p._parse()
File "PubMedParser.py", line 64, in _parse
event, root = context.next()
AttributeError: '_IterParseIterator' object has no attribute 'next'
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "PubMedParser.py", line 658, in <module>
run(options.medline_path, options.clean, int(options.start), options.end, int(options.PROCESSES))
File "PubMedParser.py", line 621, in run
res = result.get()
File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
AttributeError: '_IterParseIterator' object has no attribute 'next'
是代码问题还是python3中的问题(3.5.2是我使用的版本)?
在实施Patrick Artner的建议后,会出现更多不同的错误,但不确定这是否与之前的错误相关:
/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed. Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
util.warn(msg)
/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed. Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
util.warn(msg)
/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed. Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
util.warn(msg)
/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed. Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
util.warn(msg)
/usr/lib/python3/dist-packages/apport/report.py:13: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
import fnmatch, glob, traceback, errno, sys, atexit, locale, imp
multiprocessing.pool.RemoteTraceback:
"""
Traceback (most recent call last):
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 507, in do_execute
cursor.execute(statement, parameters)
psycopg2.IntegrityError: null value in column "id" violates not-null constraint
DETAIL: Failing row contains (null, medline_444.xml, null, null, null, 2018-01-31 20:33:00.183484).
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
result = (True, func(*args, **kwds))
File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
return list(map(*args))
File "PubMedParser.py", line 594, in _start_parser
s = p._parse()
File "PubMedParser.py", line 574, in _parse
self.session.commit()
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 943, in commit
self.transaction.commit()
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 467, in commit
self._prepare_impl()
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2243, in flush
self._flush(objects)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2369, in _flush
transaction.rollback(_capture_exception=True)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
compat.reraise(exc_type, exc_value, exc_tb)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
raise value
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2333, in _flush
flush_context.execute()
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute
rec.execute(self)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute
uow
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
mapper, table, insert)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/persistence.py", line 866, in _emit_insert_statements
execute(statement, params)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 948, in execute
return meth(self, multiparams, params)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
return connection._execute_clauseelement(self, multiparams, params)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
context)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
exc_info
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
reraise(type(exception), exception, tb=exc_tb, cause=cause)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 186, in reraise
raise value.with_traceback(tb)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
context)
File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 507, in do_execute
cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in column "id" violates not-null constraint
DETAIL: Failing row contains (null, medline_444.xml, null, null, null, 2018-01-31 20:33:00.183484).
[SQL: 'INSERT INTO pubmed.tbl_xml_file (xml_file_name, doc_type_name, dtd_public_id, dtd_system_id, time_processed) VALUES (%(xml_file_name)s, %(doc_type_name)s, %(dtd_public_id)s, %(dtd_system_id)s, %(time_processed)s)'] [parameters: {'dtd_public_id': None, 'dtd_system_id': None, 'xml_file_name': 'medline_444.xml', 'doc_type_name': None, 'time_processed': datetime.datetime(2018, 1, 31, 20, 33, 0, 183484)}] (Background on this error at: http://sqlalche.me/e/gkpj)
"""
The above exception was the direct cause of the following exception:
Traceback (most recent call last):
File "PubMedParser.py", line 659, in <module>
run(options.medline_path, options.clean, int(options.start), options.end, int(options.PROCESSES))
File "PubMedParser.py", line 622, in run
res = result.get()
File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get
raise self._value
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in column "id" violates not-null constraint
DETAIL: Failing row contains (null, medline_444.xml, null, null, null, 2018-01-31 20:33:00.183484).
[SQL: 'INSERT INTO pubmed.tbl_xml_file (xml_file_name, doc_type_name, dtd_public_id, dtd_system_id, time_processed) VALUES (%(xml_file_name)s, %(doc_type_name)s, %(dtd_public_id)s, %(dtd_system_id)s, %(time_processed)s)'] [parameters: {'dtd_public_id': None, 'dtd_system_id': None, 'xml_file_name': 'medline_444.xml', 'doc_type_name': None, 'time_processed': datetime.datetime(2018, 1, 31, 20, 33, 0, 183484)}] (Background on this error at: http://sqlalche.me/e/gkpj)
答案 0 :(得分:1)
PendingDeprecationWarning:
the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
说了一切。 Python上的开发人员决定删除imp
或importlib
- 它仍然在代码库中但将被删除。转到模块文档,阅读有关如何将imp
代码翻译成importlib
代码的信息。
https://docs.python.org/3/library/imp.html
从版本3.4开始不推荐使用:imp包正在等待弃用,转而使用importlib。
这是只是警告(现在注意它,或者在将python更新为删除了imp的版本时修复它)。
搜索SO,f.e。:Problems when converting from imp to importlib in python 3.4
其他东西使程序崩溃。通过改变
可以解决https://stackoverflow.com/a/42742804/7505395的问题event, root = self.context.next()
到此:
event, root = context.__next__()
(未测试的)。