Question

我已经在Sagemaker中训练了一个基于tf.estimator的TensorFlow模型，并对其进行了部署，并且效果很好。

但是我只能以JSON格式向其发送请求。我需要发送一些大的输入张量，这看起来效率很低，并且很快就打破了InvokeEndpoints 5MB的请求限制。

是否有可能针对基于tensorflow服务的端点使用更有效的格式？

我尝试发送基于protobuf的请求：

from sagemaker.tensorflow.serving import Model 
from sagemaker.tensorflow.tensorflow_serving.apis import predict_pb2 
from sagemaker.tensorflow.predictor import tf_serializer, tf_deserializer

role = 'xxx'

model = Model('s3://xxx/tmp/artifacts/sagemaker-tensorflow-scriptmode-xxx/output/model.tar.gz', role)

predictor = model.deploy(initial_instance_count=1, instance_type='ml.c5.xlarge', endpoint_name='test-endpoint')

# this predictor has json serializer, make a new one pred = 

RealTimePredictor('test-endpoint', serializer=tf_serializer, deserializer=tf_deserializer)

req = predict_pb2.PredictRequest()

req.inputs['instances'].CopyFrom(tf.make_tensor_proto(np.zeros((4, 36, 64)), shape=(4, 36, 64)))

predictor.predict(req)

这将导致以下错误：

---------------------------------------------------------------------------
ModelError                                Traceback (most recent call last)
<ipython-input-40-5ba7f281bd0d> in <module>()
----> 1 predictor.predict(req)

~/anaconda3/envs/default/lib/python3.6/site-packages/sagemaker/predictor.py in predict(self, data, initial_args)
     76 
     77         request_args = self._create_request_args(data, initial_args)
---> 78         response = self.sagemaker_session.sagemaker_runtime_client.invoke_endpoint(**request_args)
     79         return self._handle_response(response)
     80 

~/anaconda3/envs/default/lib/python3.6/site-packages/botocore/client.py in _api_call(self, *args, **kwargs)
    355                     "%s() only accepts keyword arguments." % py_operation_name)
    356             # The "self" in this scope is referring to the BaseClient.
--> 357             return self._make_api_call(operation_name, kwargs)
    358 
    359         _api_call.__name__ = str(py_operation_name)

~/anaconda3/envs/default/lib/python3.6/site-packages/botocore/client.py in _make_api_call(self, operation_name, api_params)
    659             error_code = parsed_response.get("Error", {}).get("Code")
    660             error_class = self.exceptions.from_code(error_code)
--> 661             raise error_class(parsed_response, operation_name)
    662         else:
    663             return parsed_response

ModelError: An error occurred (ModelError) when calling the InvokeEndpoint operation: Received client error (415) from model with message "{"error": "Unsupported Media Type: application/octet-stream"}".

JSON是否是已部署的TensorFlow模型的唯一可用查询格式？

Answer 1

您看过批量转换吗？如果您实际上不需要HTTPS端点，则可以解决您的问题：

https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-batch-transform.html

比JSON更有效的方法来发送请求到Sagemaker中已部署的tensorflow模型？

1 个答案: