apport report.py迁移到python3

时间:2018-01-31 19:17:52

标签: python-3.x sqlalchemy python-multiprocessing

尝试迁移项目listed here 哪个使用多处理,我收到以下错误:

/usr/lib/python3/dist-packages/apport/report.py:13: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import fnmatch, glob, traceback, errno, sys, atexit, locale, imp
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "PubMedParser.py", line 593, in _start_parser
    s = p._parse()
  File "PubMedParser.py", line 64, in _parse
    event, root = context.next()
AttributeError: '_IterParseIterator' object has no attribute 'next'
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "PubMedParser.py", line 658, in <module>
    run(options.medline_path, options.clean, int(options.start), options.end, int(options.PROCESSES))
  File "PubMedParser.py", line 621, in run
    res = result.get()
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
AttributeError: '_IterParseIterator' object has no attribute 'next'

是代码问题还是python3中的问题(3.5.2是我使用的版本)?

在实施Patrick Artner的建议后,会出现更多不同的错误,但不确定这是否与之前的错误相关:

/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed.  Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
  util.warn(msg)
/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed.  Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
  util.warn(msg)
/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed.  Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
  util.warn(msg)
/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/crud.py:700: SAWarning: Column 'pubmed.tbl_xml_file.id' is marked as a member of the primary key for table 'pubmed.tbl_xml_file', but has no Python-side or server-side default generator indicated, nor does it indicate 'autoincrement=True' or 'nullable=True', and no explicit value is passed.  Primary key columns typically may not store NULL. Note that as of SQLAlchemy 1.1, 'autoincrement=True' must be indicated explicitly for composite (e.g. multicolumn) primary keys if AUTO_INCREMENT/SERIAL/IDENTITY behavior is expected for one of the columns in the primary key. CREATE TABLE statements are impacted by this change as well on most backends.
  util.warn(msg)
/usr/lib/python3/dist-packages/apport/report.py:13: PendingDeprecationWarning: the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses
  import fnmatch, glob, traceback, errno, sys, atexit, locale, imp
multiprocessing.pool.RemoteTraceback: 
"""
Traceback (most recent call last):
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
psycopg2.IntegrityError: null value in column "id" violates not-null constraint
DETAIL:  Failing row contains (null, medline_444.xml, null, null, null, 2018-01-31 20:33:00.183484).


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 119, in worker
    result = (True, func(*args, **kwds))
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 44, in mapstar
    return list(map(*args))
  File "PubMedParser.py", line 594, in _start_parser
    s = p._parse()
  File "PubMedParser.py", line 574, in _parse
    self.session.commit()
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 943, in commit
    self.transaction.commit()
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 467, in commit
    self._prepare_impl()
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 447, in _prepare_impl
    self.session.flush()
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2243, in flush
    self._flush(objects)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2369, in _flush
    transaction.rollback(_capture_exception=True)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/langhelpers.py", line 66, in __exit__
    compat.reraise(exc_type, exc_value, exc_tb)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 187, in reraise
    raise value
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/session.py", line 2333, in _flush
    flush_context.execute()
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/unitofwork.py", line 391, in execute
    rec.execute(self)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/unitofwork.py", line 556, in execute
    uow
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/persistence.py", line 181, in save_obj
    mapper, table, insert)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/orm/persistence.py", line 866, in _emit_insert_statements
    execute(statement, params)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 948, in execute
    return meth(self, multiparams, params)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/sql/elements.py", line 269, in _execute_on_connection
    return connection._execute_clauseelement(self, multiparams, params)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1060, in _execute_clauseelement
    compiled_sql, distilled_params
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1200, in _execute_context
    context)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1413, in _handle_dbapi_exception
    exc_info
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 203, in raise_from_cause
    reraise(type(exception), exception, tb=exc_tb, cause=cause)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/util/compat.py", line 186, in reraise
    raise value.with_traceback(tb)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/base.py", line 1193, in _execute_context
    context)
  File "/usr/local/lib/python3.5/dist-packages/sqlalchemy/engine/default.py", line 507, in do_execute
    cursor.execute(statement, parameters)
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in column "id" violates not-null constraint
DETAIL:  Failing row contains (null, medline_444.xml, null, null, null, 2018-01-31 20:33:00.183484).
 [SQL: 'INSERT INTO pubmed.tbl_xml_file (xml_file_name, doc_type_name, dtd_public_id, dtd_system_id, time_processed) VALUES (%(xml_file_name)s, %(doc_type_name)s, %(dtd_public_id)s, %(dtd_system_id)s, %(time_processed)s)'] [parameters: {'dtd_public_id': None, 'dtd_system_id': None, 'xml_file_name': 'medline_444.xml', 'doc_type_name': None, 'time_processed': datetime.datetime(2018, 1, 31, 20, 33, 0, 183484)}] (Background on this error at: http://sqlalche.me/e/gkpj)
"""

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "PubMedParser.py", line 659, in <module>
    run(options.medline_path, options.clean, int(options.start), options.end, int(options.PROCESSES))
  File "PubMedParser.py", line 622, in run
    res = result.get()
  File "/usr/lib/python3.5/multiprocessing/pool.py", line 608, in get
    raise self._value
sqlalchemy.exc.IntegrityError: (psycopg2.IntegrityError) null value in column "id" violates not-null constraint
DETAIL:  Failing row contains (null, medline_444.xml, null, null, null, 2018-01-31 20:33:00.183484).
 [SQL: 'INSERT INTO pubmed.tbl_xml_file (xml_file_name, doc_type_name, dtd_public_id, dtd_system_id, time_processed) VALUES (%(xml_file_name)s, %(doc_type_name)s, %(dtd_public_id)s, %(dtd_system_id)s, %(time_processed)s)'] [parameters: {'dtd_public_id': None, 'dtd_system_id': None, 'xml_file_name': 'medline_444.xml', 'doc_type_name': None, 'time_processed': datetime.datetime(2018, 1, 31, 20, 33, 0, 183484)}] (Background on this error at: http://sqlalche.me/e/gkpj)

1 个答案:

答案 0 :(得分:1)

PendingDeprecationWarning: 
the imp module is deprecated in favour of importlib; see the module's documentation for alternative uses

说了一切。 Python上的开发人员决定删除impimportlib - 它仍然在代码库中但将被删除。转到模块文档,阅读有关如何将imp代码翻译成importlib代码的信息。

  

https://docs.python.org/3/library/imp.html
  从版本3.4开始不推荐使用:imp包正在等待弃用,转而使用importlib。

这是只是警告(现在注意它,或者在将python更新为删除了imp的版本时修复它)。

搜索SO,f.e。:Problems when converting from imp to importlib in python 3.4

其他东西使程序崩溃。通过改变

可以解决https://stackoverflow.com/a/42742804/7505395的问题
event, root = self.context.next() 

到此:

event, root = context.__next__()

(未测试的)。