Question

tf服务正常退出而没有错误消息

系统信息

OS平台和发行版（例如Linux Ubuntu 16.04）：ReaHat EL6

TensorFlow Serving从（源或二进制）安装：使用bazel 0.18.0的源

TensorFlow服务版本：1.12.0

描述问题

我在RHEL 6.9中使用bazel编译tf-serving，并使用以下命令启动它：

./ model_servers / tensorflow_model_server --model_config_file =。/ data / models.conf --rest_api_port = 8502

models.conf：

model_config_list: {
  config: {
    name: "model_1",
base_path:"/search/work/tf_serving_bin/tensorflow_serving/data/model_data/model_1",
    model_platform: "tensorflow",
    model_version_policy: {
      latest: {
        num_versions: 1
      }
    }
  }
}

使用C ++的客户端，并使用libCurl请求提供tf服务的REST api，但是，tf服务通常会在几分钟内出现异常退出而没有错误消息。

当我的客户服务请求localhost tf-serving时，该问题经常发生。但是，客户端服务要求在其他计算机上进行tf-serving，这个问题不会出现，qps <100。

我检查内存，cpu空闲等...没有发现问题。所以，这很奇怪。

export export TF_CPP_MIN_VLOG_LEVEL = 1，也没有错误/严重消息。

源代码/日志

2019-01-09 09:28:35.118183: I tensorflow_serving/model_servers/server_core.cc:461] Adding/updating models.
2019-01-09 09:28:35.118259: I tensorflow_serving/model_servers/server_core.cc:558] (Re-)adding model: app_ks_nfm_1
2019-01-09 09:28:35.227383: I tensorflow_serving/core/basic_manager.cc:739] Successfully reserved resources to load servable {name: app_ks_nfm_1 version: 201901072359}
2019-01-09 09:28:35.227424: I tensorflow_serving/core/loader_harness.cc:66] Approving load for servable version {name: app_ks_nfm_1 version: 201901072359}
2019-01-09 09:28:35.227443: I tensorflow_serving/core/loader_harness.cc:74] Loading servable version {name: app_ks_nfm_1 version: 201901072359}
2019-01-09 09:28:35.227492: I external/org_tensorflow/tensorflow/contrib/session_bundle/bundle_shim.cc:363] Attempting to load native SavedModelBundle in bundle-shim from: /search/work/bazel-bin-serving/tensorflow_serving/data/model_data/app_ks_nfm_1/201901072359
2019-01-09 09:28:35.227530: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:31] Reading SavedModel from: /search/work/bazel-bin-serving/tensorflow_serving/data/model_data/app_ks_nfm_1/201901072359
2019-01-09 09:28:35.256712: I external/org_tensorflow/tensorflow/cc/saved_model/reader.cc:54] Reading meta graph with tags { serve }
2019-01-09 09:28:35.267728: I external/org_tensorflow/tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: SSE4.1 SSE4.2 AVX AVX2 FMA
2019-01-09 09:28:35.313087: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:162] Restoring SavedModel bundle.
2019-01-09 09:28:38.797633: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:138] Running MainOp with key legacy_init_op on SavedModel bundle.
2019-01-09 09:28:38.803984: I external/org_tensorflow/tensorflow/cc/saved_model/loader.cc:259] SavedModel load for tags { serve }; Status: success. Took 3570131 microseconds.
2019-01-09 09:28:38.804027: I tensorflow_serving/servables/tensorflow/saved_model_warmup.cc:83] No warmup data file found at /search/work/bazel-bin-serving/tensorflow_serving/data/model_data/app_ks_nfm_1/201901072359/assets.extra/tf_serving_warmup_requests
2019-01-09 09:28:38.804148: I tensorflow_serving/core/loader_harness.cc:86] Successfully loaded servable version {name: app_ks_nfm_1 version: 201901072359}
2019-01-09 09:28:38.831860: I tensorflow_serving/model_servers/server.cc:286] Running gRPC ModelServer at 0.0.0.0:8500 ...
[warn] getaddrinfo: address family for nodename not supported
2019-01-09 09:28:38.865243: I tensorflow_serving/model_servers/server.cc:302] Exporting HTTP/REST API at:localhost:8502 ...
[evhttp_server.cc : 237] RAW: Entering the event loop ...

Answer 1

这不是异常退出。这表明服务器已准备好接收推理请求。

为澄清起见，请找到以下说明：

docker run --runtime=nvidia -p 8501:8501 \
  --mount type=bind,\ source=/tmp/tfserving/serving/tensorflow_serving/servables/tensorflow/testdata/saved_model_half_plus_two_gpu,\
  target=/models/half_plus_two \
  -e MODEL_NAME=half_plus_two -t tensorflow/serving:latest-gpu &

这将在nvidia-docker运行时中运行docker容器，启动TensorFlow服务模型服务器，绑定REST API端口8501，并将我们所需的模型从主机映射到容器中期望模型的位置。我们还将模型的名称作为环境变量进行传递，这在查询模型时很重要。

提示：查询模型之前，请务必等待直到看到如下消息，表明服务器已准备好接收请求：

2018-07-27 00:07:20.773693: I tensorflow_serving/model_servers/main.cc:333]
Exporting HTTP/REST API at:localhost:8501 ...

该消息后，只需按Enter键，即可使用以下命令查询模型

curl -d '{"instances": [1.0, 2.0, 5.0]}' \
  -X POST http://localhost:8501/v1/models/half_plus_two:predict

有关更多信息，请参见以下链接：

https://www.tensorflow.org/tfx/serving/docker#gpu_serving_example

Answer 2

原因：短连接产品大量的TCP Status'TIME_WAIT'，可用的Linux系统文件句柄被占用。

tf服务异常退出而没有错误消息

系统信息

描述问题

源代码/日志

2 个答案: