Question

我的目标是在C ++应用程序中运行tensorrt优化张量流图。我正在使用tensorflow 1.8和tensorrt 4.使用python api我能够优化图形并看到一个不错的性能提升。

尝试在c ++中运行图表失败，并出现以下错误：

Not found: Op type not registered 'TRTEngineOp' in binary running on e15ff5301262. Make sure the Op and Kernel are registered in the binary running in this process.

其他非张量图表有效。我和python api有类似的错误，但是通过导入tensorflow.contrib.tensorrt来解决它。从错误我相当肯定内核和操作系统没有注册，但是在构建tensorflow之后，我不知道如何在应用程序中这样做。在侧面说明我不能使用bazel但我需要使用cmake。到目前为止，我链接到libtensorflow_cc.so和libtensorflow_framework.so。

有人可以帮我吗？谢谢！

更新：使用c或c ++ api加载_trt_engine_op.so不会在加载时抛出错误，但无法运行

Invalid argument: No OpKernel was registered to support Op 'TRTEngineOp' with these attrs.  Registered devices: [CPU,GPU], Registered kernels:
  <no registered kernels>

     [[Node: my_trt_op3 = TRTEngineOp[InT=[DT_FLOAT, DT_FLOAT], OutT=[DT_FLOAT, DT_FLOAT, DT_FLOAT, DT_FLOAT], input_nodes=["tower_0/down_0/conv_0/Conv2D-0-TransposeNHWCToNCHW-LayoutOptimizer", "tower_0/down_0/conv_skip/Conv2D-0-TransposeNHWCToNCHW-LayoutOptimizer"], output_nodes=["tower_0/down_0/conv_skip/Relu", "tower_0/down_1/conv_skip/Relu", "tower_0/down_2/conv_skip/Relu", "tower_0/down_3/conv_skip/Relu"], serialized_engine="\220{I\000...00\000\000"](tower_0/down_0/conv_0/Conv2D-0-TransposeNHWCToNCHW-LayoutOptimizer, tower_0/down_0/conv_skip/Conv2D-0-TransposeNHWCToNCHW-LayoutOptimizer)]]

Answer 1

另一种解决错误问题的方法＆＃34;未找到：操作类型未注册＆＃39; TRTEngineOp＆＃39;＆＃34;在Tensorflow 1.8：

1）在文件tensorflow/contrib/tensorrt/BUILD中，添加包含以下内容的新部分：

cc_library(
name = "trt_engine_op_kernel_cc",
srcs = [
    "kernels/trt_calib_op.cc",
    "kernels/trt_engine_op.cc",
    "ops/trt_calib_op.cc",
    "ops/trt_engine_op.cc",
    "shape_fn/trt_shfn.cc",
],
hdrs = [
    "kernels/trt_calib_op.h",
    "kernels/trt_engine_op.h",
    "shape_fn/trt_shfn.h",
],
copts = tf_copts(),
visibility = ["//visibility:public"],
deps = [
    ":trt_logging",
    ":trt_plugins",
    ":trt_resources",
    "//tensorflow/core:gpu_headers_lib",
    "//tensorflow/core:lib_proto_parsing",
    "//tensorflow/core:stream_executor_headers_lib",
] + if_tensorrt([
    "@local_config_tensorrt//:nv_infer",
]) + tf_custom_op_library_additional_deps(),
alwayslink = 1,  # buildozer: disable=alwayslink-with-hdrs
)

2）将//tensorflow/contrib/tensorrt:trt_engine_op_kernel_cc作为依赖项添加到您要构建的相应BAZEL项目

PS：无需使用TF_LoadLibrary

加载库_trt_engine_op.so

Answer 2

对于Tensorflow r1.8，下面在两个BUILD文件中显示的附加内容以及使用独石选项构建libtensorflow_cc.so的情况对我而言有效。

diff --git a/tensorflow/BUILD b/tensorflow/BUILD
index cfafffd..fb8eb31 100644
--- a/tensorflow/BUILD
+++ b/tensorflow/BUILD
@@ -525,6 +525,8 @@ tf_cc_shared_object(
         "//tensorflow/cc:scope",
         "//tensorflow/cc/profiler",
         "//tensorflow/core:tensorflow",
+        "//tensorflow/contrib/tensorrt:trt_conversion",
+        "//tensorflow/contrib/tensorrt:trt_engine_op_kernel",
     ],
 )

diff --git a/tensorflow/contrib/tensorrt/BUILD b/tensorflow/contrib/tensorrt/BUILD
index fd3582e..a6566b9 100644
--- a/tensorflow/contrib/tensorrt/BUILD
+++ b/tensorflow/contrib/tensorrt/BUILD
@@ -76,6 +76,8 @@ cc_library(
     srcs = [
         "kernels/trt_calib_op.cc",
         "kernels/trt_engine_op.cc",
+        "ops/trt_calib_op.cc",
+        "ops/trt_engine_op.cc",
     ],
     hdrs = [
         "kernels/trt_calib_op.h",
@@ -86,6 +88,7 @@ cc_library(
     deps = [
         ":trt_logging",
         ":trt_resources",
+        ":trt_shape_function",
         "//tensorflow/core:gpu_headers_lib",
         "//tensorflow/core:lib_proto_parsing",
         "//tensorflow/core:stream_executor_headers_lib",

Answer 3

正如您所提到的，当您将ls_attr <- sapply(my_list, attributes) search_attr <- matrix( c( 'c', 'b', 2:3), ncol = nrow(ls_attr), byrow = TRUE ) my_list [ apply( matrix( ls_attr %in% search_attr, nrow = nrow(ls_attr), byrow = FALSE ), 2, all) ]添加到依赖项列表时，它应该可以正常工作。目前Tensorflow-TensorRT集成仍在进行中，可能仅适用于python API;对于C ++，您需要从//tensorflow/contrib/tensorrt:trt_engine_op_kernel致电ConvertGraphDefToTensorRT()进行转换。

如果您有任何问题，请与我们联系。

Answer 4

以下是此问题的发现（以及某种解决方案）（Tensorflow 1.8.0，TensorRT 3.0.4）：

我想将tensorrt支持包含到库中，该库从给定的*.pb文件加载图形。

只是将//tensorflow/contrib/tensorrt:trt_engine_op_kernel添加到我的Bazel BUILD文件中并没有为我做到这一点。我仍然收到一条消息，表明Ops未注册：

2018-05-21 12:22:07.286665: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "TRTCalibOp" device_type: "GPU"') for unknown op: TRTCalibOp
2018-05-21 12:22:07.286856: E tensorflow/core/framework/op_kernel.cc:1242] OpKernel ('op: "TRTEngineOp" device_type: "GPU"') for unknown op: TRTEngineOp
2018-05-21 12:22:07.296024: E tensorflow/examples/tf_inference_lib/cTfInference.cpp:56] Not found: Op type not registered 'TRTEngineOp' in binary running on ***. 
Make sure the Op and Kernel are registered in the binary running in this process.

解决方法是，我必须使用C_API在我的C ++代码中加载Ops库（tf_custom_op_library）：

#include "tensorflow/c/c_api.h"
...
TF_Status status = TF_NewStatus();
TF_LoadLibrary("_trt_engine_op.so", status);

为bazel目标_trt_engine_op.so创建共享对象//tensorflow/contrib/tensorrt:python/ops/_trt_engine_op.so：

bazel build --config=opt --config=cuda --config=monolithic \
     //tensorflow/contrib/tensorrt:python/ops/_trt_engine_op.so

现在我只需要确保_trt_engine_op.so在需要时可用，例如LD_LIBRARY_PATH。

如果有人有想法，如何以更优雅的方式做到这一点（为什么我们要制作2件必须构建的文物？我们只能拥有一件吗？），我很高兴每一个建议。

<强> tldr

将//tensorflow/contrib/tensorrt:trt_engine_op_kernel添加为您要构建的相应BAZEL项目的依赖项
使用C-API在代码中加载ops-library _trt_engine_op.so。

Answer 5

解决方案：添加导入

从tensorflow.python.compiler.tensorrt导入trt_convert为trt

链接讨论：https://github.com/tensorflow/tensorflow/issues/26525

Answer 6

这是我的解决方案，张量流为1.14。在您的BUILD文件中，exp，tensorflow / examples / your_workspace / BUILD：

在tf_cc_binary中：

scrs= [...,"//tensorflow/compiler/tf2tensorrt:ops/trt_engine_op.cc"]
deps=[...,"//tensorflow/compiler/tf2tensorrt:trt_op_kernels"]

带TensorRT的C ++ Tensorflow API

6 个答案: