这是错误报告:
[Sat Jan 21 12:07:50.518064 2017] [mpm_event:notice] [pid 8928:tid 139985251233280] AH00491: caught SIGTERM, shutting down
[Sat Jan 21 12:07:51.400192 2017] [wsgi:warn] [pid 9240:tid 139692690144768] mod_wsgi: Compiled for Python/2.7.11.
[Sat Jan 21 12:07:51.400309 2017] [wsgi:warn] [pid 9240:tid 139692690144768] mod_wsgi: Runtime using Python/2.7.12+.
[Sat Jan 21 12:07:51.405286 2017] [mpm_event:notice] [pid 9240:tid 139692690144768] AH00489: Apache/2.4.18 (Ubuntu) mod_wsgi/4.3.0 Python/2.7.12+ configured -- resuming normal operations
[Sat Jan 21 12:07:51.405340 2017] [core:notice] [pid 9240:tid 139692690144768] AH00094: Command line: '/usr/sbin/apache2'
[Sat Jan 21 12:08:02.453647 2017] [wsgi:error] [pid 9245:tid 139692552484608] Invoked on Sat Jan 21 12:07:57 UTC 2017 with arguments: -loadClassifier /var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/classifiers/english.all.3class.distsim.crf.ser.gz -textFile /tmp/tmpb53bnN -outputFormat slashTags -tokenizerFactory edu.stanford.nlp.process.WhitespaceTokenizer -tokenizerOptions "tokenizeNLs=false" -encoding utf8
[Sat Jan 21 12:08:02.454348 2017] [wsgi:error] [pid 9245:tid 139692552484608] tokenizerFactory=edu.stanford.nlp.process.WhitespaceTokenizer
[Sat Jan 21 12:08:02.454365 2017] [wsgi:error] [pid 9245:tid 139692552484608] tokenizerOptions="tokenizeNLs=false"
[Sat Jan 21 12:08:02.454381 2017] [wsgi:error] [pid 9245:tid 139692552484608] loadClassifier=/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/classifiers/english.all.3class.distsim.crf.ser.gz
[Sat Jan 21 12:08:02.454388 2017] [wsgi:error] [pid 9245:tid 139692552484608] encoding=utf8
[Sat Jan 21 12:08:02.454392 2017] [wsgi:error] [pid 9245:tid 139692552484608] textFile=/tmp/tmpb53bnN
[Sat Jan 21 12:08:02.454397 2017] [wsgi:error] [pid 9245:tid 139692552484608] outputFormat=slashTags
[Sat Jan 21 12:08:02.454415 2017] [wsgi:error] [pid 9245:tid 139692552484608]
[Sat Jan 21 12:08:02.472037 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] [2017-01-21 12:08:02,461] ERROR in app: Exception on / [GET]
[Sat Jan 21 12:08:02.472087 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] Traceback (most recent call last):
[Sat Jan 21 12:08:02.472093 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1982, in wsgi_app
[Sat Jan 21 12:08:02.472099 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] response = self.full_dispatch_request()
[Sat Jan 21 12:08:02.472104 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1614, in full_dispatch_request
[Sat Jan 21 12:08:02.472109 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] rv = self.handle_user_exception(e)
[Sat Jan 21 12:08:02.472113 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1517, in handle_user_exception
[Sat Jan 21 12:08:02.472118 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] reraise(exc_type, exc_value, tb)
[Sat Jan 21 12:08:02.472122 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1612, in full_dispatch_request
[Sat Jan 21 12:08:02.472126 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] rv = self.dispatch_request()
[Sat Jan 21 12:08:02.472131 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/flask/app.py", line 1598, in dispatch_request
[Sat Jan 21 12:08:02.472135 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] return self.view_functions[rule.endpoint](**req.view_args)
[Sat Jan 21 12:08:02.472140 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/var/www/FlaskApp/FlaskApp/__init__.py", line 34, in homepage
[Sat Jan 21 12:08:02.472163 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] NERtagger.tag(testy.split())
[Sat Jan 21 12:08:02.472169 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/nltk/tag/stanford.py", line 71, in tag
[Sat Jan 21 12:08:02.472173 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] return sum(self.tag_sents([tokens]), [])
[Sat Jan 21 12:08:02.472177 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/nltk/tag/stanford.py", line 94, in tag_sents
[Sat Jan 21 12:08:02.472182 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] stdout=PIPE, stderr=PIPE)
[Sat Jan 21 12:08:02.472186 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] File "/usr/local/lib/python2.7/dist-packages/nltk/internals.py", line 134, in java
[Sat Jan 21 12:08:02.472190 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] raise OSError('Java command failed : ' + str(cmd))
[Sat Jan 21 12:08:02.472195 2017] [wsgi:error] [pid 9245:tid 139692552484608] [client 141.69.97.198:10001] OSError: Java command failed : [u'/usr/bin/java', '-mx1000m', '-cp', '/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/stanford-ner.jar:/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/stanford-ner-3.7.0-javadoc.jar:/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/stanford-ner-3.7.0-sources.jar:/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/stanford-ner-3.7.0.jar:/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/lib/jollyday-0.4.9.jar:/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/lib/joda-time.jar:/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/lib/stanford-ner-resources.jar', 'edu.stanford.nlp.ie.crf.CRFClassifier', '-loadClassifier', '/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/classifiers/english.all.3class.distsim.crf.ser.gz', '-textFile', '/tmp/tmpb53bnN', '-outputFormat', 'slashTags', '-tokenizerFactory', 'edu.stanford.nlp.process.WhitespaceTokenizer', '-tokenizerOptions', '"tokenizeNLs=false"', '-encoding', 'utf8']
服务器配置: 512 MB内存/ 20 GB磁盘/ NYC3 - Ubuntu 16.10 x64
我认为问题正在发生,因为JVM无法运行只有512 MB内存的文件。当我尝试从我的apache服务器执行时(Java和Python都安装在其中)会出现问题:
testy = request.args.get('query')
path_to_model = "/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/classifiers/english.all.3class.distsim.crf.ser.gz"
path_to_jar = "/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/stanford-ner.jar"
NERtagger = StanfordNERTagger(path_to_model, path_to_jar)
tagging=NERtagger.tag(testy.split())
english.all.3class.distsim.crf.ser.gz是一个大文件,大小为38 MB。这段代码在我的笔记本电脑中完美运行,它有i7 / 16 gb内存。但是我的在线服务器没有运行,只有512 MB的内存。如果我尝试使用尺寸为48 kb的自定义小尺寸ner-customModel.ser.gz,代码将在我的服务器中运行。
testy = request.args.get('query')
path_to_model = "/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/custom/ner-customModel.ser.gz"
path_to_jar = "/var/www/FlaskApp/FlaskApp/StanFordNer/stanford-ner-2016-10-31/stanford-ner.jar"
NERtagger = StanfordNERTagger(path_to_model, path_to_jar)
tagging=NERtagger.tag(testy.split())
我认为问题不在于斯坦福大学图书馆,因为我的笔记本电脑python安装中的两个代码都工作正常,后面的代码也在我的服务器上运行。只有当我尝试执行大型服务器stanford-ner-2016-10-31.ser.gz(38 MB)时才产生错误。我认为在那种情况下JVM尝试使用更多内存。有什么方法可以解决这个问题吗?