我正在使用uWSGI(2.0.11.2)和Python(3.4.3)在Ubuntu 14.04上为我的Pyramid(1.5.7)应用程序提供服务。我注意到我的uWSGI日志中出现了与unicode解码相关的错误:
#
# one of the situations when exception is raised is
# when SQLAlchemy (which has set INFO logging level)
# tries to write an SQL statement containing unicode charater
# into log file
#
2016-02-26 16:01:38,734 INFO [sqlalchemy.engine.base.Engine][b'uWSGIWorker5Core0'] BEGIN (implicit)
2016-02-26 16:01:38,735 INFO [sqlalchemy.engine.base.Engine][b'uWSGIWorker5Core0'] SELECT * FROM staging WHERE company_name = %(company_name_1)s AND time = %(time_1)s AND ship_name = %(ship_name_1)s
# exact place (missing line) where SQLAlchemy is trying to print out
# query parameters, which in this case include unicode character
--- Logging error ---
Traceback (most recent call last):
File "/usr/lib/python3.4/logging/__init__.py", line 980, in emit
stream.write(msg)
UnicodeEncodeError: 'ascii' codec can't encode character '\xfa' in position 132: ordinal not in range(128)
Call stack:
File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/sqltap/wsgi.py", line 42, in __call__
return self.app(environ, start_response)
File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/pyramid/router.py", line 242, in __call__
response = self.invoke_subrequest(request, use_tweens=True)
#
# the stack continues...
# full stack here -> https://bpaste.net/show/8e12af790372
#
File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1010, in _execute_clauseelement
compiled_sql, distilled_params
File "/home/mk/.virtualenvs/api/lib/python3.4/site-packages/sqlalchemy/engine/base.py", line 1100, in _execute_context
sql_util._repr_params(parameters, batches=10)
Unable to print the message and arguments - possible formatting error.
Use the traceback above to help find the error.
我还注意到将相同的行写入Pyramid生成的日志文件(不涉及uWSGI)工作正常,没有任何错误,并且正确插入了unicode字符。
我正在使用此命令运行uWSGI:
/usr/local/bin/uwsgi --emperor /etc/uwsgi/vassals
在vassals
文件夹中,我已经从我的Pyramid应用程序中对符号链接了uWSGI配置,它看起来像这样:
[uwsgi]
host = %h
username = mk
project_name = api
project_root = /shared/projects/python/%(project_name)
env = PYTHONIOENCODING=UTF-8
; this env var is generated based on host name
env = APP_INI_FILE=develop.ini
; folders config
home_folder = /home/%(username)
virtualenv_folder = %(home_folder)/.virtualenvs/%(project_name)
logs_folder = %(home_folder)/logs/%(project_name)
chdir = %(project_root)
socket = %(project_root)/%(project_name).sock
pidfile = %(project_root)/%(project_name).pid
virtualenv = %(virtualenv_folder)
daemonize = %(logs_folder)/uwsgi.log
; core stuff
master = true
vacuum = true
processes = 5
enable-threads = true
; socket conf
chmod-socket = 666 # invoking the One
chown-socket = %(username)
uid = %(username)
gid = %(username)
; log conf
log-reopen = true
logfile-chown = %(username)
logfile-chmod = 644
; app conf
module = wsgi:application
harakiri = 120
max-requests = 500
post-buffering = 1
paste = config:%p
paste-logger = $p
Pyramid的配置文件,其中定义了所有日志记录,如下所示:
###
# app configuration
# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/environment.html
###
[DEFAULT]
home_dir = /home/mk
[app:main]
use = egg:api
pyramid.reload_templates = false
pyramid.debug_authorization = false
pyramid.debug_notfound = false
pyramid.debug_routematch = false
pyramid.default_locale_name = en
sqlalchemy.url = postgresql://XXX:YYY@12.13.14.15:5432/ze_database?client_encoding=utf8
[server:main]
use = egg:waitress#main
host = 0.0.0.0
port = 6543
###
# logging configuration
# http://docs.pylonsproject.org/projects/pyramid/en/1.5-branch/narr/logging.html
###
[loggers]
keys = root, sqlalchemy
[handlers]
keys = console, debuglog
[formatters]
keys = generic, short
[logger_root]
level = DEBUG
handlers = console, debuglog
[logger_sqlalchemy]
level = INFO
handlers =
qualname = sqlalchemy.engine
# "level = INFO" logs SQL queries.
# "level = DEBUG" logs SQL queries and results.
# "level = WARN" logs neither. (Recommended for production systems.)
[handler_console]
class = StreamHandler
args = (sys.stderr,)
level = DEBUG
formatter = generic
[handler_debuglog]
class = handlers.RotatingFileHandler
args = ('%(home_dir)s/logs/api/pyramid_debug.log', 'a', 1024000000, 10)
level = DEBUG
formatter = generic
[formatter_generic]
format = %(asctime)s %(levelname)-5.5s [%(name)s][%(threadName)s] %(message)s
[formatter_short]
format = %(asctime)s %(message)s
最后,我的Pyramid的wsgi.py
文件非常简单:
import os
from pyramid.paster import get_app, setup_logging
here = os.path.dirname(os.path.abspath(__file__))
conf = os.path.join(here, os.environ.get('APP_INI_FILE')) # APP_INI_FILE variable is set in uwsgi.ini
setup_logging(conf)
application = get_app(conf, 'main')
基本上,我将应用程序的日志重定向到stderr
(或stdout
,就我注意到的情况而言都是一样的)我也将其写在一个单独的文件中{{1} }) 在同一时间。在我的例子中,pyramid_debug.log
是uWSGI守护程序的日志文件,这就是错误发生的地方。
虽然stderr
及相关变量在系统上设置为LC_ALL
,但我也尝试使用各种与本地化相关的环境变量,并在我的Pyramid的wsgi应用程序中明确设置它们,但是很少幸运 - 例如,在uWSGI配置中仅设置en_EN.UTF-8
变量解决了我本地计算机上的问题,但在我部署后没有解决问题。
我明显的问题是 - 在这种情况下如何正确处理uWSGI在日志文件中写入unicode字符?
答案 0 :(得分:0)
修改File "/usr/lib/python3.4/logging/__init__.py", line 980
并更改
stream.write(msg)
到
stream.write(msg.encode('utf-8'))
最有可能的是,流类型的更改方式不会破坏您的UTF-8编码功能,但实际上是因为Pythonics。 (似乎无论世界大声尖叫“UTF-8”,Python都忽略了所有人。)
例如,如果您正在处理文件,则会忽略您尝试的所有环境变量:
# test.py
import sys
sys.stdout.write(u'\u00f6\n')
试验:
max% python test.py
ö
max% python test.py > f
Traceback (most recent call last):
File "test.py", line 2, in <module>
sys.stdout.write(u'\u00f6\n')
UnicodeEncodeError: 'ascii' codec can't encode character u'\xf6' in position 0: ordinal not in range(128)