问题

Question

我正在研究基本的Tensorflow服务示例。我正在关注MNIST示例，除了分类，我想使用 numpy数组来预测另一个 numpy数组。

要做到这一点，我首先训练我的神经网络

x = tf.placeholder("float", [None, n_input],name ="input_values")

weights = {
    'encoder_h1': tf.Variable(tf.random_normal([n_input, n_hidden_1])),
    'encoder_h2': tf.Variable(tf.random_normal([n_hidden_1, n_hidden_2])),
    'encoder_h3': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_3])),
    'decoder_h1': tf.Variable(tf.random_normal([n_hidden_3, n_hidden_2])),
    'decoder_h2': tf.Variable(tf.random_normal([n_hidden_2, n_hidden_1])),
    'decoder_h3': tf.Variable(tf.random_normal([n_hidden_1, n_input])),
}
biases = {
    'encoder_b1': tf.Variable(tf.random_normal([n_hidden_1])),
    'encoder_b2': tf.Variable(tf.random_normal([n_hidden_2])),
    'encoder_b3': tf.Variable(tf.random_normal([n_hidden_3])),
    'decoder_b1': tf.Variable(tf.random_normal([n_hidden_2])),
    'decoder_b2': tf.Variable(tf.random_normal([n_hidden_1])),
    'decoder_b3': tf.Variable(tf.random_normal([n_input])),
}

# Building the encoder
def encoder(x):
    # Encoder Hidden layer with sigmoid activation #1
    layer_1 = tf.nn.tanh(tf.matmul(x, weights['encoder_h1'])+biases['encoder_b1'])
    print(layer_1.shape)
    # Decoder Hidden layer with sigmoid activation #2
    layer_2 = tf.nn.tanh(tf.matmul(layer_1, weights['encoder_h2'])+biases['encoder_b2'])
    print(layer_2.shape)
    # Layer 3
    layer_3 = tf.nn.tanh(tf.matmul(layer_2, weights['encoder_h3'])+biases['encoder_b3'])
    print(layer_3.shape)
    return layer_3


# Building the decoder
def decoder(x):
    # Encoder Hidden layer with sigmoid activation #1
    layer_1 = tf.nn.tanh(tf.matmul(x, weights['decoder_h1'])+biases['decoder_b1'])
    print(layer_1.shape)
    # Decoder Hidden layer with sigmoid activation #2
    layer_2 = tf.nn.tanh(tf.matmul(layer_1, weights['decoder_h2'])+biases['decoder_b2'])
    # Layer 3
    layer_3 = tf.nn.tanh(tf.matmul(layer_2, weights['decoder_h3'])+biases['decoder_b3'])
    return layer_3

# Construct model
encoder_op = encoder(x)
decoder_op = decoder(encoder_op)

# Prediction
y = decoder_op



# Objective functions
y_ = tf.placeholder("float", [None,n_input],name="predict")

接下来有人在这里建议我保存了我的网络就好了..

import os
import sys

from tensorflow.python.saved_model import builder as saved_model_builder
from tensorflow.python.saved_model import utils
from tensorflow.python.saved_model import tag_constants, signature_constants
from tensorflow.python.saved_model.signature_def_utils_impl import     build_signature_def, predict_signature_def
from tensorflow.contrib.session_bundle import exporter

with tf.Session() as sess:
# Initialize variables
    sess.run(init)

    # Restore model weights from previously saved model
    saver.restore(sess, model_path)
    print("Model restored from file: %s" % save_path)

    export_path = '/tmp/AE_model/6'
    print('Exporting trained model to', export_path)
    builder = tf.saved_model.builder.SavedModelBuilder(export_path)


    signature = predict_signature_def(inputs={'inputs': x},
                                  outputs={'outputs': y})

    builder.add_meta_graph_and_variables(sess=sess,
                                         tags=[tag_constants.SERVING],
                                         signature_def_map={'predict': signature})

    builder.save()


    print 'Done exporting!'

接下来，我按照说明在localhost：9000上运行我的服务器

bazel build //tensorflow_serving/model_servers:tensorflow_model_server

我设置了服务器

bazel-bin/tensorflow_serving/model_servers/tensorflow_model_server --port=9000 --model_base_path=/tmp/AE_model/

问题

现在我想编写一个程序，这样我就可以将eclipse中的C ++程序中的Mat向量（我使用了很多库）传递给我的服务器，这样我就可以进行某种预测。

我首先考虑使用inception_client.cc作为参考。但是，似乎我需要Bazel来编译它，因为我无法在任何地方找到prediction_service.grpc.pb.h :(

所以似乎我唯一的另一个选择是使用python调用脚本我得到以下输出：

<grpc.beta._client_adaptations._Rendezvous object at 0x7f9bcf8cb850>

对此问题的任何帮助都将不胜感激。

谢谢。

编辑：

我重新安装了protobuf和grpc并按照建议运行命令：

我的命令有点不同，我不得不在我的服务文件夹之外使用它（在Ubuntu 14.04中）。

sudo protoc -I=serving -I serving/tensorflow --grpc_out=. --plugin=protoc-gen-grpc=`which grpc_cpp_plugin` serving/tensorflow_serving/apis/*.proto

这会生成.gprc.pb.h文件，然后将它们拖到/ apis /文件夹中，错误就会消失。现在我收到了错误

/tensorflow/third_party/eigen3/unsupported/Eigen/CXX11/Tensor:1:42: fatal error: unsupported/Eigen/CXX11/Tensor: No such file or directory

即使此文件确实存在。任何建议表示赞赏。

谢谢@subzero！

编辑2

通过更新到最新的Eigen版本并从源码构建，我能够解决Eigen的问题。接下来我指向/ usr / local / include / eigen3 /

之后我遇到了tensorflow库的问题。我通过使用lababidi的建议生成libtensorflow_cc.so库来解决这些问题。 https://github.com/tensorflow/tensorflow/issues/2412

我有最后一期。一切似乎都很好，除非我收到错误：

对`tensorflow :: serving :: PredictRequest ::〜PredictRequest（）＆＃39;

的未定义引用

似乎我缺少链接器或库。有谁知道我错过了什么？

Answer 1

自定义客户端和服务器的示例：

要添加到张量流模型的服务器代码：

import grpc
from concurrent import futures
import python_pb2
import python_pb2_grpc

class PythonServicer(python_pb2_grpc.PythonServicer):


    def makePredictions(self, request, context):


        items = eval(str(request.items)) #Receives the input values for the model as a string and evaluates them into an array to be passed to tensorflow

        x_feed = items

        targetEval_out = sess.run(confidences, feed_dict={x:x_feed}) #"confidences" is the output of my model, replace it for the appropriate function from your model


        out = str(targetEval_out.tolist()) #The model output is then put into string format to be passed back to the client. It has to be reformatted on the other end, but this method was easier to implement

        return python_pb2.value(name=out)


print("server online")
MAX_MESSAGE_LENGTH = 4 * 1024 * 1024 #can be edited to allow for larger amount of data to be transmitted per message. This can be helpful for making large numbers of predictions at once.
server = grpc.server(futures.ThreadPoolExecutor(max_workers=10), 
options=[('grpc.max_send_message_length', MAX_MESSAGE_LENGTH), (
'grpc.max_receive_message_length', MAX_MESSAGE_LENGTH)])
python_pb2_grpc.add_PythonServicer_to_server(
PythonServicer(), server)
server.add_insecure_port('[::]:50051')
server.start()

客户端C ++代码：

#include <grpc/grpc.h>
#include <grpc++/channel.h>
#include <grpc++/client_context.h>
#include <grpc++/create_channel.h>
#include <grpc++/security/credentials.h>
#include "python.grpc.pb.h"

using grpc::Channel;
using grpc::ClientContext;
using grpc::ClientReader;
using grpc::ClientReaderWriter;
using grpc::ClientWriter;
using grpc::Status;
using python::request;
using python::value;
using python::Python;

using namespace std;


unsigned MAX_MESSAGE_LENGTH = 4 * 1024 * 1024; //can be edited to allow for larger amount of data to be transmitted per message. This can be helpful for making large numbers of predictions at once.
grpc::ChannelArguments channel_args;
channel_args.SetMaxReceiveMessageSize(MAX_MESSAGE_LENGTH);
channel_args.SetMaxSendMessageSize(MAX_MESSAGE_LENGTH);

shared_ptr<Channel> channel = CreateCustomChannel("localhost:50051", grpc::InsecureChannelCredentials(),channel_args);
unique_ptr<python::Python::Stub>stub = python::Python::NewStub(channel);

request r;
r.set_items(dataInputString); //The input data should be a string that can be parsed to a python array, for example "[[1.0,2.0,3.0],[4.0,5.0,6.0]]"
//The server code was made to be able to make multiple predictions at once, hence the multiple data arrays 
value val;
ClientContext context;

Status status = stub->makePredictions(&context, r, &val);

cout << val.name() << "\n"; //This prints the returned model prediction

python.proto代码：

syntax = "proto3";


package python;

service Python {

    rpc makePredictions(request) returns (value) {}


}

message request {
  string items = 1;
}


message value {
  string name = 1;
}

我不确定这些代码段是否可以自行运行，因为我刚从当前项目中复制了相关代码。但希望这对任何需要张量流客户端和服务器的人来说都是一个很好的起点。

Answer 2

您要查找的pb.h文件是通过在this file上运行protc生成的。

您可以按照here所述的说明生成头文件并自行使用。在任何情况下，您运行的Basel构建都应该在构建目录中生成此文件，您可以设置eclipse项目以使用这些包含路径来编译C客户端。

Answer 3

它不是C ++而且它并不完全是您正在寻找的东西，但我们在这里有一个预测客户端，您可以根据需要使用它。

https://github.com/epigramai/tfserving_predict_client/

它基于这里的代码，https://github.com/tobegit3hub/tensorflow_template_application，我认为可能有一个C ++客户端可以在那里使用。

服务客户端

问题

编辑：

编辑2

3 个答案: