如何使用自定义protobuf构建tensorflow 1.13.1？

时间：2019-04-08 17:38:02

标签： tensorflow protocol-buffers

系统信息

  • 我是否编写了自定义代码：修改bazel文件
  • OS平台和发行版：Ubuntu 16.04
  • TensorFlow从以下来源安装：
  • TensorFlow版本：1.13.1是从相应标签（https://github.com/tensorflow/tensorflow/archive/v1.13.1.tar.gz）下载的
  • Python版本：Python 3.5.2
  • Bazel版本：0.21.0
  • Protobuf版本：3.7.0（从源代码构建并放置在文件系统中的某个位置）
  • CUDA / cuDNN版本：10.0 / 7
  • GPU模型和内存：GeForce GTX 1060 Ti

上下文：默认情况下，tensorflow会构建自己的protobuf代码。尽管protobuf也被其他库使用，但它的导出符号与tensorflow提供的符号冲突。解决该问题的唯一好方法是对所有库（包括tensorflow）使用唯一且独立（即不在tensorflow中）的protobuf版本。所以我基本上需要使用目标安装版本的protobuf构建tensorflow，该版本位于文件系统中的某个位置。

问题：在构建tensorflow 1.13.1时，使用protobuf的自定义版本，该版本安装在文件系统中的某个位置（不在默认系统路径中）。更具体地说，我的问题是：需要对tensorflow的bazel文件进​​行哪些修改才能使其实现。我是bazel的新手，我对如何做感到很困惑...

这是我所做的：

1）停用protobuf的内部版本，我在.tf_configure.bazelrc中添加了以下行：

build --action_env TF_SYSTEM_LIBS="protobuf_archive"

这可以按预期工作，除了我的protobuf安装在默认系统路径中的版本太旧以至于无法解析proto3文件。无论如何，这并不是一个真正的问题，因为我想使用3.7.0版的自定义protobuf。

2）指定在哪里找到protobuf，我通过使用new_local_repository而不是tf_http_archive来更改了workspace.bzl文件。

如果我的文件系统中安装了protobuf库的路径，则@ PATH_TO_PROTOBUF @。 

    new_local_repository(
        name =  "protobuf_archive",
        path = "@PATH_TO_PROTOBUF@",
        build_file = clean_dep("//third_party/systemlibs:protobuf.BUILD"),
    )

    new_local_repository(
        name = "com_google_protobuf",
        path = "@PATH_TO_PROTOBUF@",
        system_build_file = clean_dep("//third_party/systemlibs:protobuf.BUILD"),
        system_link_files = {
            "//third_party/systemlibs:protobuf.bzl": "protobuf.bzl",
        },
    )
    new_local_repository(
        name = "com_google_protobuf_cc",
        path = "@PATH_TO_PROTOBUF@",
        system_build_file = clean_dep("//third_party/systemlibs:protobuf.BUILD"),
        system_link_files = {
            "//third_party/systemlibs:protobuf.bzl": "protobuf.bzl",
        },
    )

3）我通过更改规则使用的二进制文件，更改了位于tensorflow-1.13.1 / third_party / systemlibs中的protobuf.BUILD文件：

cc_library(
    name = "protobuf",
    hdrs = HEADERS,
    linkopts = ["@PATH_TO_PROTOBUF@/lib/libprotobuf.so"],
    visibility = ["//visibility:public"],
)

cc_library(
    name = "protobuf_headers",
    hdrs = HEADERS,
    linkopts = ["@PATH_TO_PROTOBUF@/lib/libprotobuf.so"],
    visibility = ["//visibility:public"],
)

cc_library(
    name = "protoc_lib",
    linkopts = ["@PATH_TO_PROTOBUF@/lib/libprotoc.so"],
    visibility = ["//visibility:public"],
)

genrule(
    name = "protoc",
    outs = ["protoc.bin"],
    cmd = "ln -s @PATH_TO_PROTOBUF@/bin/protoc $@",
    executable = 1,
    visibility = ["//visibility:public"],
)

这样，我以为一切都会正常，但是当我运行构建时：

ERROR: .../tensorflow-1.13.1/tensorflow/core/BUILD:2460:1: ProtoCompile tensorflow/core/lib/core/error_codes.pb.cc failed (Exit 127): protoc.bin failed: error executing command 
  (cd /home/robin/.cache/bazel/_bazel_robin/c04a70144cd329180403af87e4cbdc44/execroot/org_tensorflow && \
  exec env - \
    PATH=/bin:/usr/bin \
  bazel-out/host/genfiles/external/protobuf_archive/protoc.bin '--cpp_out=bazel-out/host/genfiles/' -I. -Iexternal/protobuf_archive -Ibazel-out/host/genfiles/external/protobuf_archive tensorflow/core/lib/core/error_codes.proto)
Execution platform: @bazel_tools//platforms:host_platform
[32 / 203] 6 actions, 5 running
    Executing genrule @protobuf_archive//:link_headers [for host]; 0s local
    ProtoCompile .../core/lib/core/error_codes.pb.cc [for host]; 0s local
    Compiling tensorflow/core/platform/default/logging.cc [for host]; 0s local
    ProtoCompile tensorflow/core/example/example.pb.cc [for host]; 0s local
    Executing genrule @local_config_cuda//cuda:cuda-include; 0s local
    [-----] Writing file external/llvm/llvm-tblgen-2.params [for host]
bazel-out/host/genfiles/external/protobuf_archive/protoc.bin: error while loading shared libraries: libprotoc.so.18: cannot open shared object file: No such file or directory

显然protoc失败只是因为加载器找不到libprotoc。

4）因此，该解决方案对我而言微不足道，只需将LD_LIBRARY_PATH设置为足以自动找到libprotoc.so即可。不幸的是，以下解决方案均无效：

A）直接设置LD_LIBRARY_PATH 

export LD_LIBRARY_PATH=@PATH_TO_PROTOBUF@/lib
bazel build //tensorflow:tensorflow_cc.So

B）通过.tf_configure.bazelrc设置LD_LIBRARY_PATH：

build --action_env LD_LIBRARY_PATH="@PATH_TO_PROTOBUF@/lib"

输出与以前完全相同，因此我的第一句话是LD_LIBRARY_PATH不导出（据我了解）。可以解释这是因为：

exec env - \
    PATH=/bin:/usr/bin \
bazel-out/host/genfiles/external/protobuf_archive/protoc.bin '--cpp_out=bazel-out/host/genfiles/'

不包含

这样的表达式 
LD_LIBRARY_PATH=@PATH_TO_PROTOBUF@/lib/

我搜索了很长时间，但没有找到解决该问题的任何方法（我测试了很多，但没有任何效果）...也许这是我使用的Bazel版本的限制？不幸的是，我不能仅仅因为tensorflow 1.13.1禁止使用更新的Bazel版本。

所以现在我不知道该怎么做...我想解决方案是对tensorflow的bazel项目文件进行更多修改...

2 个答案:

答案 0 :(得分：1)

对于那些感兴趣的人，我发现了一个不太优雅但似乎可行的解决方案：

1）编写脚本call_proto.sh 

export LD_LIBRARY_PATH=@TARGET_PROTOBUF_LIB_DIR@:$LD_LIBRARY_PATH
@TARGET_PROTOC_EXECUTABLE@ $@

将@ TARGET_PROTOBUF_LIB_DIR @和@ TARGET_PROTOC_EXECUTABLE @替换为适当的值（库目录包含protoc库以及protoc可执行文件的完整路径）。

2）来自我上一个示例中的tensorflow-1.13.1 / third_party / systemlibs / protobuf.BUILD替换为：

genrule(
    name = "protoc",
    outs = ["protoc.bin"],
    cmd = "ln -s @PATH_TO_PROTOBUF@/bin/protoc $@",
    executable = 1,
    visibility = ["//visibility:public"],
)

作者

genrule(
    name = "protoc",
    outs = ["protoc.bin"],
    cmd = "ln -s @PROTO_CALL_SCRIPT@ $@",
    executable = 1,
    visibility = ["//visibility:public"],
)

使用@ PROTO_CALL_SCRIPT @指向上一个脚本文件的路径...

这确实解决了将LD_LIBRARY_PATH传递给协议调用的问题...

不幸的是，现在又出现了另一个问题：

ERROR: /home/robin/.cache/bazel/_bazel_robin/c04a70144cd329180403af87e4cbdc44/external/protobuf_archive/BUILD.bazel:44:1: declared output 'external/protobuf_archive/google/protobuf/any.pb.h' is a dangling symbolic link
[21 / 86] 7 actions, 6 running
    Executing genrule @protobuf_archive//:link_headers [for host]; 0s local
    Executing genrule @local_config_cuda//cuda:cuda-include; 0s local
    @com_google_absl//absl/base:dynamic_annotations; 0s local
    Executing genrule @local_config_cuda//cuda:cuda-lib [for host]; 0s local
    ProtoCompile tensorflow/core/example/example.pb.cc [for host]; 0s local
    ProtoCompile tensorflow/core/example/example.pb.cc; 0s local
    [-----] //tensorflow/cc:ops/candidate_sampling_ops_gen_cc
ERROR: /home/robin/.cache/bazel/_bazel_robin/c04a70144cd329180403af87e4cbdc44/external/protobuf_archive/BUILD.bazel:44:1: declared output 'external/protobuf_archive/google/protobuf/any.proto' is a dangling symbolic link
[21 / 89] 7 actions, 6 running
    Executing genrule @protobuf_archive//:link_headers [for host]; 0s local
    Executing genrule @local_config_cuda//cuda:cuda-include; 0s local
    @com_google_absl//absl/base:dynamic_annotations; 0s local
    Executing genrule @local_config_cuda//cuda:cuda-lib [for host]; 0s local
    ProtoCompile tensorflow/core/example/example.pb.cc [for host]; 0s local
    ProtoCompile tensorflow/core/example/example.pb.cc; 0s local
    [-----] //tensorflow/cc:ops/candidate_sampling_ops_gen_cc
ERROR: /home/robin/.cache/bazel/_bazel_robin/c04a70144cd329180403af87e4cbdc44/external/protobuf_archive/BUILD.bazel:44:1: declared output 'external/protobuf_archive/google/protobuf/arena.h' is a dangling symbolic link
[21 / 89] 7 actions, 6 running
...

protobuf的所有标头出现相同的错误消息is a dangling symbolic link。

实际上，在相应的protobu_archive文件夹中，我有悬挂的符号链接，例如：

any.pb.h -> /usr/include/google/protobuf/any.pb.h

据我了解，问题出在link_headers类别：

genrule(
    name = "link_headers",
    outs = HEADERS,
    cmd = """
      echo "OUTS=$(OUTS) include=$(INCLUDEDIR)"
      for i in $(OUTS); do
        f=$${i#$(@D)/}
        mkdir -p $(@D)/$${f%/*}
        ln -sf $(INCLUDEDIR)/$$f $(@D)/$$f
      done
    """,
)

由于INCLUDEDIR值（/usr/include），所有生成的符号链接均无效。

解决方案是修改thios规则，使其指向有效的包含文件夹：

genrule(
    name = "link_headers",
    outs = HEADERS,
    cmd = """
      for i in $(OUTS); do
        f=$${i#$(@D)/}
        mkdir -p $(@D)/$${f%/*}
        ln -sf @TARGET_PROTOBUF_INCLUDE@/$$f $(@D)/$$f
      done
    """,
)

使用@ TARGET_PROTOBUF_INCLUDE @到protobuf的路径包括文件系统中的dir ...现在正确生成了符号链接，并且悬挂的符号链接的错误也消失了。

我有一个错误报告，指出protobuf的某些头文件是未知的……要解决此问题，解决方案只是在protobuf.BUILD中为HEADERS生成足够的值。我将HEADERS的值设置为：

HEADERS = [
    "google/protobuf/any.h",
    "google/protobuf/any.pb.h",
    "google/protobuf/any.proto",
    "google/protobuf/api.pb.h",
    "google/protobuf/api.proto",
    "google/protobuf/arena.h",
    "google/protobuf/arena_impl.h",
    "google/protobuf/arenastring.h",
    "google/protobuf/compiler/code_generator.h",
    "google/protobuf/compiler/command_line_interface.h",
    "google/protobuf/compiler/cpp/cpp_generator.h",
    "google/protobuf/compiler/csharp/csharp_generator.h",
    "google/protobuf/compiler/csharp/csharp_names.h",
    "google/protobuf/compiler/importer.h",
    "google/protobuf/compiler/java/java_generator.h",
    "google/protobuf/compiler/java/java_names.h",
    "google/protobuf/compiler/js/js_generator.h",
    "google/protobuf/compiler/js/well_known_types_embed.h",
    "google/protobuf/compiler/objectivec/objectivec_generator.h",
    "google/protobuf/compiler/objectivec/objectivec_helpers.h",
    "google/protobuf/compiler/parser.h",
    "google/protobuf/compiler/php/php_generator.h",
    "google/protobuf/compiler/plugin.h",
    "google/protobuf/compiler/plugin.pb.h",
    "google/protobuf/compiler/plugin.proto",
    "google/protobuf/compiler/python/python_generator.h",
    "google/protobuf/compiler/ruby/ruby_generator.h",
    "google/protobuf/descriptor.h",
    "google/protobuf/descriptor.pb.h",
    "google/protobuf/descriptor.proto",
    "google/protobuf/descriptor_database.h",
    "google/protobuf/duration.pb.h",
    "google/protobuf/duration.proto",
    "google/protobuf/dynamic_message.h",
    "google/protobuf/empty.pb.h",
    "google/protobuf/empty.proto",
    "google/protobuf/extension_set.h",
    "google/protobuf/extension_set_inl.h",
    "google/protobuf/field_mask.pb.h",
    "google/protobuf/field_mask.proto",
    "google/protobuf/generated_enum_reflection.h",
    "google/protobuf/generated_enum_util.h",
    "google/protobuf/generated_message_reflection.h",
    "google/protobuf/generated_message_table_driven.h",
    "google/protobuf/generated_message_util.h",
    "google/protobuf/has_bits.h",
    "google/protobuf/implicit_weak_message.h",
    "google/protobuf/inlined_string_field.h",
    "google/protobuf/io/coded_stream.h",
    "google/protobuf/io/gzip_stream.h",
    "google/protobuf/io/printer.h",
    "google/protobuf/io/strtod.h",
    "google/protobuf/io/tokenizer.h",
    "google/protobuf/io/zero_copy_stream.h",
    "google/protobuf/io/zero_copy_stream_impl.h",
    "google/protobuf/io/zero_copy_stream_impl_lite.h",
    "google/protobuf/map.h",
    "google/protobuf/map_entry.h",
    "google/protobuf/map_entry_lite.h",
    "google/protobuf/map_field.h",
    "google/protobuf/map_field_inl.h",
    "google/protobuf/map_field_lite.h",
    "google/protobuf/map_type_handler.h",
    "google/protobuf/message.h",
    "google/protobuf/message_lite.h",
    "google/protobuf/metadata.h",
    "google/protobuf/metadata_lite.h",
    "google/protobuf/parse_context.h",
    "google/protobuf/port.h",
    "google/protobuf/port_def.inc",
    "google/protobuf/port_undef.inc",
    "google/protobuf/reflection.h",
    "google/protobuf/reflection_ops.h",
    "google/protobuf/repeated_field.h",
    "google/protobuf/service.h",
    "google/protobuf/source_context.pb.h",
    "google/protobuf/source_context.proto",
    "google/protobuf/struct.pb.h",
    "google/protobuf/struct.proto",
    "google/protobuf/stubs/bytestream.h",
    "google/protobuf/stubs/callback.h",
    "google/protobuf/stubs/casts.h",
    "google/protobuf/stubs/common.h",
    "google/protobuf/stubs/fastmem.h",
    "google/protobuf/stubs/hash.h",
    "google/protobuf/stubs/logging.h",
    "google/protobuf/stubs/macros.h",
    "google/protobuf/stubs/mutex.h",
    "google/protobuf/stubs/once.h",
    "google/protobuf/stubs/platform_macros.h",
    "google/protobuf/stubs/port.h",
    "google/protobuf/stubs/status.h",
    "google/protobuf/stubs/stl_util.h",
    "google/protobuf/stubs/stringpiece.h",
    "google/protobuf/stubs/strutil.h",
    "google/protobuf/stubs/template_util.h",
    "google/protobuf/text_format.h",
    "google/protobuf/timestamp.pb.h",
    "google/protobuf/timestamp.proto",
    "google/protobuf/type.pb.h",
    "google/protobuf/type.proto",
    "google/protobuf/unknown_field_set.h",
    "google/protobuf/util/delimited_message_util.h",
    "google/protobuf/util/field_comparator.h",
    "google/protobuf/util/field_mask_util.h",
    "google/protobuf/util/json_util.h",
    "google/protobuf/util/message_differencer.h",
    "google/protobuf/util/time_util.h",
    "google/protobuf/util/type_resolver.h",
    "google/protobuf/util/type_resolver_util.h",
    "google/protobuf/wire_format.h",
    "google/protobuf/wire_format_lite.h",
    "google/protobuf/wire_format_lite_inl.h",
    "google/protobuf/wrappers.pb.h",
    "google/protobuf/wrappers.proto",
]

这只是protobuf include dir中包含的文件的列表。经过检查后，我现在可以说我的特定protobuf安装的所有符号链接都已放入bazel-genfiles/external/protobuf_archive。

我当时非常有信心，但是可能因为我有一个新的错误而太过自信了：

ERROR: .../tensorflow-1.13.1/tensorflow/stream_executor/BUILD:18:1: C++ compilation of rule '//tensorflow/stream_executor:dnn_proto_cc_impl' failed (Exit 1): crosstool_wrapper_driver_is_not_gcc failed: error executing command 
  (cd /home/robin/.cache/bazel/_bazel_robin/c04a70144cd329180403af87e4cbdc44/execroot/org_tensorflow && \
  exec env - \
    PATH=/bin:/usr/bin \
    PWD=/proc/self/cwd \
  external/local_config_cuda/crosstool/clang/bin/crosstool_wrapper_driver_is_not_gcc -MD -MF bazel-out/host/bin/tensorflow/stream_executor/_objs/dnn_proto_cc_impl/dnn.pb.pic.d '-frandom-seed=bazel-out/host/bin/tensorflow/stream_executor/_objs/dnn_proto_cc_impl/dnn.pb.pic.o' -iquote . -iquote bazel-out/host/genfiles -iquote bazel-out/host/bin -iquote external/protobuf_archive -iquote bazel-out/host/genfiles/external/protobuf_archive -iquote bazel-out/host/bin/external/protobuf_archive '-std=c++11' -Wno-builtin-macro-redefined '-D__DATE__="redacted"' '-D__TIMESTAMP__="redacted"' '-D__TIME__="redacted"' -fPIC -U_FORTIFY_SOURCE '-D_FORTIFY_SOURCE=1' -fstack-protector -Wall -fno-omit-frame-pointer -no-canonical-prefixes -fno-canonical-system-headers -DNDEBUG -g0 -O2 -ffunction-sections -fdata-sections -g0 '-march=native' -g0 -Wno-unknown-warning-option -Wno-unused-but-set-variable -Wno-sign-compare -c bazel-out/host/genfiles/tensorflow/stream_executor/dnn.pb.cc -o bazel-out/host/bin/tensorflow/stream_executor/_objs/dnn_proto_cc_impl/dnn.pb.pic.o)
Execution platform: @bazel_tools//platforms:host_platform
[255 / 752] 9 actions running
    @local_config_cuda//cuda:cuda-include; 5s local
    @com_google_absl//absl/base:base; 0s local
    Compiling tensorflow/stream_executor/dnn.pb.cc [for host]; 0s local
    Executing genrule @jpeg//:simd_x86_64_assemblage23 [for host]; 0s local
    Compiling .../costs/op_performance_data.pb.cc [for host]; 0s local
    Compiling .../core/protobuf/transport_options.pb.cc [for host]; 0s local
    Compiling .../core/protobuf/rewriter_config.pb.cc [for host]; 0s local
    Compiling tensorflow/core/framework/types.pb.cc [for host]; 0s local ...
In file included from bazel-out/host/genfiles/tensorflow/stream_executor/dnn.pb.cc:4:0:
bazel-out/host/genfiles/tensorflow/stream_executor/dnn.pb.h:10:10: fatal error: google/protobuf/port_def.inc: No such file or directory
 #include <google/protobuf/port_def.inc>
          ^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
compilation terminated.

再次遇到包含问题，但不在同一目标中。这次我真的不明白发生了什么。确实有一个选项-iquote bazel-out/host/genfiles/external/protobuf_archive传递给了呼叫，并且相应的文件夹确实包含了先前在google/protobuf/port_def.inc处生成的符号链接。

我认为问题来自传递给编译器的-iquote标志。应该是-I，不是吗？我该如何解决这个问题？

----编辑----

解决方案是为include文件夹添加include指令，如下所示：

cc_library(
    name = "protobuf",
    hdrs = HEADERS,
    includes = ["."],
    linkopts = ["@TARGET_PROTOBUF_LIB@"],
    visibility = ["//visibility:public"],
)

cc_library(
    name = "protobuf_headers",
    hdrs = HEADERS,
    includes = ["."],
    linkopts = ["@TARGET_PROTOBUF_LIB@"],
    visibility = ["//visibility:public"],
)

备注：根据bazel文档，显然使用includes = ["."],是一种反模式，但这是我发现唯一使之起作用的方法...

最后我回到第一个错误：

ERROR: /home/robin/soft/PID/pid-workspace/wrappers/tensorflow/build/1.13.1/tensorflow-1.13.1/tensorflow/core/BUILD:2489:1: Executing genrule //tensorflow/core:protos_all_proto_text_srcs failed (Exit 127): bash failed: error executing command 
  (cd /home/robin/.cache/bazel/_bazel_robin/c04a70144cd329180403af87e4cbdc44/execroot/org_tensorflow && \
  exec env - \
    PATH=/bin:/usr/bin \
  /bin/bash -c 'source external/bazel_tools/tools/genrule/genrule-setup.sh; bazel-out/host/bin/tensorflow/tools/proto_text/gen_proto_text_functions bazel-out/host/genfiles/tensorflow/core tensorflow/core/ tensorflow/core/example/example.proto tensorflow/core/example/feature.proto tensorflow/core/framework/allocation_description.proto tensorflow/core/framework/api_def.proto tensorflow/core/framework/attr_value.proto tensorflow/core/framework/cost_graph.proto tensorflow/core/framework/device_attributes.proto tensorflow/core/framework/function.proto tensorflow/core/framework/graph.proto tensorflow/core/framework/graph_transfer_info.proto tensorflow/core/framework/iterator.proto tensorflow/core/framework/kernel_def.proto tensorflow/core/framework/log_memory.proto tensorflow/core/framework/node_def.proto tensorflow/core/framework/op_def.proto tensorflow/core/framework/reader_base.proto tensorflow/core/framework/remote_fused_graph_execute_info.proto tensorflow/core/framework/resource_handle.proto tensorflow/core/framework/step_stats.proto tensorflow/core/framework/summary.proto tensorflow/core/framework/tensor.proto tensorflow/core/framework/tensor_description.proto tensorflow/core/framework/tensor_shape.proto tensorflow/core/framework/tensor_slice.proto tensorflow/core/framework/types.proto tensorflow/core/framework/variable.proto tensorflow/core/framework/versions.proto tensorflow/core/protobuf/config.proto tensorflow/core/protobuf/cluster.proto tensorflow/core/protobuf/debug.proto tensorflow/core/protobuf/device_properties.proto tensorflow/core/protobuf/queue_runner.proto tensorflow/core/protobuf/rewriter_config.proto tensorflow/core/protobuf/tensor_bundle.proto tensorflow/core/protobuf/saver.proto tensorflow/core/util/event.proto tensorflow/core/util/memmapped_file_system.proto tensorflow/core/util/saved_tensor_slice.proto tensorflow/core/lib/core/error_codes.proto tensorflow/tools/proto_text/placeholder.txt')
Execution platform: @bazel_tools//platforms:host_platform
[834 / 1,712] 9 actions running
    @com_google_absl//absl/strings:strings; 1s local
    @com_google_absl//absl/strings:strings; 0s local
    @com_google_absl//absl/strings:strings; 0s local
    @com_google_absl//absl/types:optional; 0s local
    @com_google_absl//absl/types:optional; 0s local
    @com_google_absl//absl/strings:strings; 0s local
    Executing genrule //tensorflow/core:protos_all_proto_text_srcs; 0s local
    //tensorflow/core:protos_all_proto_text_srcs; 0s local ...
bazel-out/host/bin/tensorflow/tools/proto_text/gen_proto_text_functions: error while loading shared libraries: libprotobuf.so.18: cannot open shared object file: No such file or directory

原因与第一次相同：没有给命令LD_LIBRARY_PATH。但是那个时候我不知道我应该修改哪个文件来执行此操作...更一般而言，我担心这种情况会在构建过程调用使用libprotobuf / libprotoc的可执行文件时出现。所以我的问题是：是否有一种方法可以通过修改tensorflow项目中的一个或多个文件为任何目标彻底解决此问题（将LD_LIBRARY_PATH传递给环境）？还是bazel内部存在问题？

答案 1 :(得分：0)

希望以下答案可以帮助某人：

最后看来，这是一个可以通过修改tensorflow bazel文件解决的问题。在文件tensorflow.bzl中，用以下方法修改函数tf_generate_proto_text_sources：

def tf_generate_proto_text_sources(name, srcs_relative_dir, srcs, protodeps = [], deps = [], visibility = None):
    out_hdrs = (
        [
            p.replace(".proto", ".pb_text.h")
            for p in srcs
        ] + [p.replace(".proto", ".pb_text-impl.h") for p in srcs]
    )
    out_srcs = [p.replace(".proto", ".pb_text.cc") for p in srcs]
    native.genrule(
        name = name + "_srcs",
        srcs = srcs + protodeps + [clean_dep("//tensorflow/tools/proto_text:placeholder.txt")],
        outs = out_hdrs + out_srcs,
        visibility = visibility,
        cmd =
            "LD_LIBRARY_PATH=@CONFIG_LIBRARY_PATH@ " +
            "$(location //tensorflow/tools/proto_text:gen_proto_text_functions) " +
            "$(@D) " + srcs_relative_dir + " $(SRCS)",
        tools = [
            clean_dep("//tensorflow/tools/proto_text:gen_proto_text_functions"),
        ],
    )

    native.filegroup(
        name = name + "_hdrs",
        srcs = out_hdrs,
        visibility = visibility,
    )

    native.cc_library(
        name = name,
        srcs = out_srcs,
        hdrs = out_hdrs,
        visibility = visibility,
        deps = deps,
    )

@ CONFIG_LIBRARY_PATH @是一个LD_LIBRARY_PATH值，其中包含protobuf lib 目录的路径。

相关问题
最新问题