Question

对于Tensorflow和TensorRT来说，它们还很陌生，我很难将现有的冻结图转换为tensorRT图。我认为我拥有的代码无法成功转换图形。在Nvidia Jetson Nano上运行。

我尝试遵循此处显示的准则：https://docs.nvidia.com/deeplearning/frameworks/tf-trt-user-guide/index.html#using-frozengraph

def load_object_detection_model(self):
        # Load TensorFlow object detection model

        gpu_options = tf.GPUOptions(per_process_gpu_memory_fraction=.5)

        EXPORTED_OBJECT_DETECTION_MODEL = 'frozen_model_x.pb'
        self.graph_obj = tf.Graph()
        with self.graph_obj.as_default():
            od_graph_def = tf.GraphDef()
            with tf.gfile.GFile(EXPORTED_OBJECT_DETECTION_MODEL, 'rb') as fid:
                serialized_graph = fid.read()
                od_graph_def.ParseFromString(serialized_graph)
                tf.import_graph_def(od_graph_def, name='')

         # Optimize Graph with TensorRT

        trt_graph = trt.create_inference_graph(
            input_graph_def=od_graph_def,
            outputs=['num_detections', 'detection_boxes', 'detection_scores', 'detection_classes'],
            max_batch_size=1,
            max_workspace_size_bytes=4000000000,
            precision_mode='FP16')

        print('reading graph')

        output_node = tf.import_graph_def(
            trt_graph,
            return_elements=['num_detections', 'detection_boxes', 'detection_scores', 'detection_classes'])

        self.graph_obj = output_node # Replace frozen graph with optimized graph 

        print('converted graph')

我得到的错误输出是：“在load_object_detection_model中 ops = self.graph_obj.get_operations（） AttributeError：“列表”对象没有属性“ get_operations””，该属性对应于以下代码：

# get handles to objects in object detection graph
        ops = self.graph_obj.get_operations()
        all_tensor_names = {output.name for op in ops for output in op.outputs}
        self.tensor_dict = {}
        for key in [
            'num_detections', 'detection_boxes', 'detection_scores',
            'detection_classes', 'detection_masks'
        ]:
            tensor_name = key + ':0'
            if tensor_name in all_tensor_names:
                self.tensor_dict[key] = self.graph_obj.get_tensor_by_name(tensor_name)

        self.obj_image_tensor = self.graph_obj.get_tensor_by_name('image_tensor:0')
        self.logger.debug('created object detection model graph from {}'.format(EXPORTED_OBJECT_DETECTION_MODEL))

        # create session for object detection
        self.sess_obj = tf.Session(graph=self.graph_obj)
        self.logger.debug('created object detection model session')

（上面的代码在上一个代码段之后）。

运行Ubuntu 18.04，Python 3.6.8，TensorFlow 1.13.1。下面的TensorRT详细信息：

ii  graphsurgeon-tf                            5.0.6-1+cuda10.0                                arm64        GraphSurgeon for TensorRT package
ii  libnvinfer-dev                             5.0.6-1+cuda10.0                                arm64        TensorRT development libraries and headers
ii  libnvinfer-samples                         5.0.6-1+cuda10.0                                all          TensorRT samples and documentation
ii  libnvinfer5                                5.0.6-1+cuda10.0                                arm64        TensorRT runtime libraries
ii  python-libnvinfer                          5.0.6-1+cuda10.0                                arm64        Python bindings for TensorRT
ii  python-libnvinfer-dev                      5.0.6-1+cuda10.0                                arm64        Python development package for TensorRT
ii  python3-libnvinfer                         5.0.6-1+cuda10.0                                arm64        Python 3 bindings for TensorRT
ii  python3-libnvinfer-dev                     5.0.6-1+cuda10.0                                arm64        Python 3 development package for TensorRT
ii  tensorrt                                   5.0.6.3-1+cuda10.0                              arm64        Meta package of TensorRT
ii  uff-converter-tf                           5.0.6-1+cuda10.0                                arm64        UFF converter for TensorRT package

Answer 1

由于pyCUDA，Jetson平台不支持TensorRT Python API。但是，python解析器运行良好。以下是一些供您参考的替代方法：

您可以使用Cython包装TensorRT C ++代码，以便可以从python调用它们。有关更多详细信息，请参阅Cython的文档。

在Jetson Nano上有一个示例可能会有所帮助：Running TensorRT Optimized GoogLeNet on Jetson Nano

将冻结图转换为TRT图时，Jetson Nano上的TensorRT错误

1 个答案: