启动地图作业时,我收到此错误。
ERROR 2015-05-11 06:03:45,719 webapp2.py:1528] __init__() got an unexpected keyword argument '_user_agent'
Traceback (most recent call last):
File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1511, in __call__
rv = self.handle_exception(request, response, e)
File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1505, in __call__
rv = self.router.dispatch(request, response)
File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1253, in default_dispatcher
return route.handler_adapter(request, response)
File "/home/rshah/google_appengine/lib/webapp2-2.3/webapp2.py", line 1076, in __call__
handler = self.handler(request, response)
File "/home/rshah/work/python/web/fourtop/mapreduce/base_handler.py", line 85, in __init__
_user_agent=self._DEFAULT_USER_AGENT))
File "/home/rshah/google_appengine/google/appengine/datastore/datastore_rpc.py", line 105, in positional_wrapper
return wrapped(*args, **kwds)
TypeError: __init__() got an unexpected keyword argument '_user_agent'
以下是我启动地图功能的方法。
control.start_map(name='Export Device Health Logs',
handler_spec='data_process.process_health_logs',
reader_spec='mapreduce.input_readers.DatastoreInputReader',
mapper_parameters={"input_reader":{'entity_kind': 'models.DeviceHealth','email':user.email,'to_date':to_date,'from_date':from_date},'output_writer': {'bucket_name': bucket_name,
'content_type': 'text/plain'}},
shard_count=1,
output_writer_spec='mapreduce.output_writers.GoogleCloudStorageConsistentOutputWriter',
mapreduce_parameters={'done_callback':'/tasks/mapreduce/done/export_health_data',
'done_callback_queue':'mapreduce-done'})
*处理程序代码: - 此代码读取设备运行状况模型的每一行,并为其准备CSV文件。 在完成对数据模型行的迭代之后,我们准备CSV文件并通过电子邮件发送给用户
def process_health_logs(health_logs):
ctx = context.get()
params = ctx.mapreduce_spec.mapper.params
to_date = params.get('to_date')
from_date = params.get('from_date')
to_date = datetime.datetime.strptime(str(to_date),'%Y-%m-%d').date()
from_date = datetime.datetime.strptime(str(from_date),'%Y-%m-%d').date()
error_date = health_logs.error_date if health_logs.error_date else health_logs.added_date
date = error_date.date()
if date >= from_date and date <= to_date:
added_date = time_util.utc_to_local(error_date, health_logs.place.timezone)
device_id = health_logs.device_id
user = health_logs.user.email
place = health_logs.place.name
unique_device_id = health_logs.unique_device_id
error_type = health_logs.error_type
error_level = health_logs.error_level
error_description = health_logs.error_description
row_data=[
added_date,
device_id,
user,
place,
unique_device_id,
error_type,
error_level,
error_description
]
row_data = [gtools.str_or_empty(x).encode('utf-8').replace(',', ';') for x in row_data]
yield ','.join(row_data) + '\n'
答案 0 :(得分:1)
很可能您正在使用旧的cloudstorage模块,与mapreduce模块不兼容。