内存错误初始模型flask api with gunicorn在AWS上运行

时间:2018-06-11 04:29:54

标签: python amazon-web-services tensorflow gunicorn flask-restful

我正在使用gunicorn进行性能测试初始模型flask api(创建多个进程) 错误:OOM在分配具有形状[800,1280,3]的张量并在/ job上键入float:localhost / replica:0 / task:0 / device:GPU:0 by allocator GPU_0_bfc      [[Node:Cast = CastDstT = DT_FLOAT,SrcT = DT_UINT8,_device =“/ job:localhost / replica:0 / task:0 / device:GPU:0”]] 提示:如果要在OOM发生时查看已分配的张量列表,请将report_tensor_allocations_upon_oom添加到RunOptions以获取当前分配信息。

由op'Cast'引起,定义于:   文件“/ usr / local / bin / gunicorn”,第11行,in     sys.exit(的run())   文件“/usr/local/lib/python3.5/dist-packages/gunicorn/app/wsgiapp.py”,第61行,在运行中     WSGIApplication(“%(prog)s [OPTIONS] [APP_MODULE]”)。run()   文件“/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py”,第223行,在运行中     super(应用程序,自我).run()   运行文件“/usr/local/lib/python3.5/dist-packages/gunicorn/app/base.py”,第72行     仲裁器(个体经营).RUN()   运行文件“/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py”,第212行     self.manage_workers()   在manage_workers中输入文件“/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py”,第545行     self.spawn_workers()   在spawn_workers中输入文件“/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py”,第616行     self.spawn_worker()   spawn_worker中的文件“/usr/local/lib/python3.5/dist-packages/gunicorn/arbiter.py”,第583行     worker.init_process()   在init_process中的文件“/usr/local/lib/python3.5/dist-packages/gunicorn/workers/base.py”,第134行     self.run()   运行文件“/usr/local/lib/python3.5/dist-packages/gunicorn/workers/sync.py”,第124行     self.run_for_one(超时)   在run_for_one中输入文件“/usr/local/lib/python3.5/dist-packages/gunicorn/workers/sync.py”,第68行     self.accept(听众)   文件“/usr/local/lib/python3.5/dist-packages/gunicorn/workers/sync.py”,第30行,接受     self.handle(监听器,客户端,地址)   文件“/usr/local/lib/python3.5/dist-packages/gunicorn/workers/sync.py”,第135行,句柄     self.handle_request(listener,req,client,addr)   在handle_request中输入文件“/usr/local/lib/python3.5/dist-packages/gunicorn/workers/sync.py”,第176行     respiter = self.wsgi(environ,resp.start_response)   文件“/usr/local/lib/python3.5/dist-packages/flask/app.py”,1997年,致电     return self.wsgi_app(environ,start_response)   在wsgi_app中输入文件“/usr/local/lib/python3.5/dist-packages/flask/app.py”,第1982行     response = self.full_dispatch_request()   在full_dispatch_request中输入文件“/usr/local/lib/python3.5/dist-packages/flask/app.py”,第1612行     rv = self.dispatch_request()   在dispatch_request中输入文件“/usr/local/lib/python3.5/dist-packages/flask/app.py”,第1598行     return self.view_functionsrule.endpoint   在classify_bulk中输入“/home/ubuntu/cv_workspace/computer_vision_services.py”,第1480行     input_operation,output_operation,tf_session = sess)   在classifyImageInSess中输入文件“/home/ubuntu/cv_workspace/src/apis/ImgClassification.py”,第228行     t = read_tensor_from_image_file(file_name)   在read_tensor_from_image_file中输入文件“/home/ubuntu/cv_workspace/src/apis/ImgClassification.py”,第51行     float_caster = tf.cast(image_reader,tf.float32)   文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/math_ops.py”,第758行,演员表     return gen_math_ops.cast(x,base_type,name = name)   文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/ops/gen_math_ops.py”,第919行,演员表     “Cast”,x = x,DstT = DstT,name = name)   在_apply_op_helper中输入文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/op_def_library.py”,第787行     op_def = op_def)   在create_op中输入文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py”,第3160行     op_def = op_def)   在 init 中输入文件“/usr/local/lib/python3.5/dist-packages/tensorflow/python/framework/ops.py”,第1625行     self._traceback = self._graph._extract_stack()#pylint:disable = protected-access

ResourceExhaustedError(参见上面的回溯):OOM在分配具有形状[800,1280,3]的张量并在/ job上键入float:localhost / replica:0 / task:0 / device:GPU:0 by allocator GPU_0_bfc      [[Node:Cast = CastDstT = DT_FLOAT,SrcT = DT_UINT8,_device =“/ job:localhost / replica:0 / task:0 / device:GPU:0”]] 提示:如果要在OOM发生时查看已分配的张量列表,请将report_tensor_allocations_upon_oom添加到RunOptions以获取当前分配信息。

0 个答案:

没有答案