Question

在加载宽和深模型后，我能够使用要素图对一个请求对象进行预测，然后将其序列化为字符串以进行预测，如下所示 -

有没有办法可以创建一批请求对象并将它们发送给tensorflow服务器进行预测？

单一预测的代码如下所示 -

for (each feature in feature list) {
    Feature feature = null;
    feature = Feature.newBuilder().setBytesList(BytesList.newBuilder().addValue(ByteString.copyFromUtf8("dummy string"))).build();

        if (feature != null) {
            inputFeatureMap.put(name, feature);
        }

}

//Converting features(in inputFeatureMap) corresponding to one request into 'Features' Proto object

                Features features = Features.newBuilder().putAllFeature(inputFeatureMap).build();
                inputStr = Example.newBuilder().setFeatures(features).build().toByteString();
        }

        TensorProto proto = TensorProto.newBuilder()
                    .addStringVal(inputStr)
                    .setTensorShape(TensorShapeProto.newBuilder().addDim(TensorShapeProto.Dim.newBuilder().setSize(1).build()).build())
                    .setDtype(DataType.DT_STRING)
                    .build();

        PredictRequest req = PredictRequest.newBuilder()
                    .setModelSpec(ModelSpec.newBuilder()
                            .setName("your serving model name")
                            .setSignatureName("serving_default")
                            .setVersion(Int64Value.newBuilder().setValue(modelVer)))
                    .putAllInputs(ImmutableMap.of("inputs", proto))
                    .build();

        PredictResponse response = stub.predict(req);

        System.out.println(response.getOutputsMap());

我们是否有办法发送预测功能对象列表，类似于此 -

List<Features> = {某种程度上创建inputFeatureMap的数组/列表，可以将其转换为序列化字符串。}

Answer 1

对于任何在这里磕磕绊绊的人，我找到了一个简单的解决方法，使用Example proto来进行批量请求。我将从this问题中借用代码并为批处理修改它。

Features features =
    Features.newBuilder()
        .putFeature("Attribute1", feature("A12"))
        .putFeature("Attribute2", feature(12))
        .putFeature("Attribute3", feature("A32"))
        .putFeature("Attribute4", feature("A40"))
        .putFeature("Attribute5", feature(7472))
        .putFeature("Attribute6", feature("A65"))
        .putFeature("Attribute7", feature("A71"))
        .putFeature("Attribute8", feature(1))
        .putFeature("Attribute9", feature("A92"))
        .putFeature("Attribute10", feature("A101"))
        .putFeature("Attribute11", feature(2))
        .putFeature("Attribute12", feature("A121"))
        .putFeature("Attribute13", feature(24))
        .putFeature("Attribute14", feature("A143"))
        .putFeature("Attribute15", feature("A151"))
        .putFeature("Attribute16", feature(1))
        .putFeature("Attribute17", feature("A171"))
        .putFeature("Attribute18", feature(1))
        .putFeature("Attribute19", feature("A191"))
        .putFeature("Attribute20", feature("A201"))
        .build();
Example example = Example.newBuilder().setFeatures(features).build();

String pfad = System.getProperty("user.dir") + "\\1511523781";
try (SavedModelBundle model = SavedModelBundle.load(pfad, "serve")) {
  Session session = model.session();
  final String xName = "input_example_tensor";
  final String scoresName = "dnn/head/predictions/probabilities:0";

  try (Tensor<String> inputBatch = Tensors.create(new byte[][] {example.toByteArray(), example.toByteArray(), example.toByteArray(), example.toByteArray()});
      Tensor<Float> output =
          session
              .runner()
              .feed(xName, inputBatch)
              .fetch(scoresName)
              .run()
              .get(0)
              .expect(Float.class)) {
    System.out.println(Arrays.deepToString(output.copyTo(new float[4][2])));
  }
}

基本上，您可以将每个示例作为byte[4][]中的对象传递，您将收到相同形状的结果float[4][2]

如何将请求对象的列表/数组传递给一个服务器调用中的tensorflow服务？

1 个答案: