如何在Tensorflow bazel build中包含contrib op

时间:2018-06-29 16:17:42

标签: c++ tensorflow bazel

我正在使用选择性注册和//tensorflow/contrib/android:libtensorflow_inference.so目标为Android进行构建:

bazel build -c opt --copt="-DSELECTIVE_REGISTRATION" --copt="-DSUPPORT_SELECTIVE_REGISTRATION" //tensorflow/contrib/android:libtensorflow_inference.so --crosstool_top=//external:android/crosstool --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cpu=x86_64

在我需要使用contrib中存在的op之前,这可以正常工作。具体来说,我需要访问ImageProjectiveTransform目标的一部分//tensorflow/contrib/image:image_ops_cc。我尝试修改//tensorflow/core:android_tensorflow_lib目标以将其添加为依赖项:

# Full TensorFlow library with operator support. Use this unless reducing
# binary size (by packaging a reduced operator set) is a concern.
cc_library(
    name = "android_tensorflow_lib",
    srcs = if_android([":android_op_registrations_and_gradients"]),
    copts = tf_copts(),
    tags = [
        "manual",
        "notap",
    ],
    visibility = ["//visibility:public"],
    deps = [
        ":android_tensorflow_lib_lite",
        ":protos_all_cc_impl",
        "//tensorflow/core/kernels:android_tensorflow_kernels",
        "//tensorflow/contrib/image:image_ops_cc",
        "//third_party/eigen3",
        "@protobuf_archive//:protobuf",
    ],
    alwayslink = 1,
)

但是现在编译失败:

In file included from external/com_googlesource_code_re2/re2/bitstate.cc:25:
In file included from external/com_googlesource_code_re2/re2/prog.h:14:
In file included from external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/include/mutex:35:
external/androidndk/ndk/sources/cxx-stl/gnu-libstdc++/4.9/include/bits/c++0x_warning.h:32:2: error: This file requires compiler and library support for the ISO C++ 2011 standard. This support is currently experimental, and must be enabled with the -std=c++11 or -std=gnu++11 compiler options.

但是如果我在bazel build调用中指定C ++ 11:

bazel build -c opt --copt="-std=c++11" --copt="-DSELECTIVE_REGISTRATION" --copt="-DSUPPORT_SELECTIVE_REGISTRATION" //tensorflow/contrib/android:libtensorflow_inference.so --crosstool_top=//external:android/crosstool --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --cpu=x86_64

然后我得到了另一个错误:

ERROR: /private/var/tmp/_bazel_json/e38619818ff94aae50ac5b3bdbbe0f32/external/png_archive/BUILD:8:1: C++ compilation of rule '@png_archive//:png' failed (Exit 1)
error: invalid argument '-std=c++11' not allowed with 'C/ObjC'
Target //tensorflow/contrib/android:libtensorflow_inference.so failed to build

(我不知道为什么Android版本中的任何东西都会使用Objective-C)

1 个答案:

答案 0 :(得分:1)

我还想包含ImageProjectiveTransform并遇到相同的错误。我可以通过以下步骤成功地将其包含进来,这些步骤是对建议here的过程的修改。

git clone https://github.com/tensorflow/tensorflow.git
git checkout r1.10
python tensorflow/tensorflow/python/tools/print_selective_registration_header.py --graphs PATH_TO_MODEL/model.pb > ops_to_register.h
cp ops_to_register.h tensorflow/tensorflow/core/framework/
cd tensorflow
bazel build -c opt --cxxopt="-DSELECTIVE_REGISTRATION" //tensorflow/contrib/android:libtensorflow_inference.so --host_crosstool_top=@bazel_tools//tools/cpp:toolchain --crosstool_top=//external:android/crosstool --cpu=armeabi-v7a --jobs 10 --cxxopt=-std=c++11
cp bazel-bin/tensorflow/contrib/android/libtensorflow_inference.so ANDROID_PROJECT/libs/armeabi-v7a/

但是,在运行此代码之前,必须进行以下修改。

这些更改解决了编译错误:

diff --git a/tensorflow/contrib/image/kernels/image_ops.h b/tensorflow/contrib/image/kernels/image_ops.h
index 209aa24..b8ec643 100644
--- a/tensorflow/contrib/image/kernels/image_ops.h
+++ b/tensorflow/contrib/image/kernels/image_ops.h
@@ -97,8 +97,8 @@ class ProjectiveGenerator {
   EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T
   nearest_interpolation(const DenseIndex batch, const float y, const float x,
                         const DenseIndex channel, const T fill_value) const {
-    return read_with_fill_value(batch, DenseIndex(std::round(y)),
-                                DenseIndex(std::round(x)), channel, fill_value);
+    return read_with_fill_value(batch, DenseIndex(::round(y)),
+                                DenseIndex(::round(x)), channel, fill_value);
   }

   EIGEN_DEVICE_FUNC EIGEN_ALWAYS_INLINE T
diff --git a/tensorflow/core/common_runtime/eager/BUILD b/tensorflow/core/common_runtime/eager/BUILD
index 7f28f3b..aa64220 100644
--- a/tensorflow/core/common_runtime/eager/BUILD
+++ b/tensorflow/core/common_runtime/eager/BUILD
@@ -150,7 +150,7 @@ tf_cuda_library(
     deps = select({
         "//tensorflow:android": [
             "//tensorflow/core:android_tensorflow_lib_lite",
-            "//util/hash:farmhash_fingerprint",
+            #"//util/hash:farmhash_fingerprint",
         ],
         "//conditions:default": [
             "//tensorflow/core:core_cpu_lib",
@@ -223,7 +223,7 @@ tf_cuda_library(
     ] + select({
         "//tensorflow:android": [
             "//tensorflow/core:android_tensorflow_lib_lite",
-            "//util/hash:farmhash_fingerprint",
+            #"//util/hash:farmhash_fingerprint",
         ],
         "//conditions:default": [
             "//tensorflow/core:core_cpu",

这会将缺少操作和内核的文件添加到构建中。对我来说,除了ImageProjectiveTransform之外,还包括Asin,Sin,Cos和ResizeArea。

diff --git a/tensorflow/core/kernels/BUILD b/tensorflow/core/kernels/BUILD
index 7599cf7..8ebb77e 100644
--- a/tensorflow/core/kernels/BUILD
+++ b/tensorflow/core/kernels/BUILD
@@ -4936,6 +4936,7 @@ filegroup(
 filegroup(
     name = "android_extended_ops_headers",
     srcs = [
+       "//tensorflow/contrib/image:image_ops_op_lib",
         "argmax_op.h",
         "avgpooling_op.h",
         "batch_matmul_op_impl.h",
@@ -4995,6 +4996,11 @@ filegroup(
 filegroup(
     name = "android_extended_ops_group1",
     srcs = [
+       "//tensorflow/contrib/image:image_ops_kernels",
+       "resize_area_op.cc",
+       "cwise_op_asin.cc",
+       "cwise_op_cos.cc",
+       "cwise_op_sin.cc",
         "argmax_op.cc",
         "avgpooling_op.cc",
         "batch_matmul_op_real.cc",

这将导致tf.contrib在加载冻结图之前注册:

diff --git a/tensorflow/python/tools/print_selective_registration_header.py b/tensorflow/python/tools/print_selective_registration_header.py
index 21d7de0..923ad76 100644
--- a/tensorflow/python/tools/print_selective_registration_header.py
+++ b/tensorflow/python/tools/print_selective_registration_header.py
@@ -40,6 +40,8 @@ import sys

 from tensorflow.python.platform import app
 from tensorflow.python.tools import selective_registration_header_lib
+from tensorflow import contrib
+contrib.resampler

 FLAGS = None

This显示了如何查找缺少操作项的* .cc文件,以便可以将其添加到BUILD中。

cd <tensorflow_repo>/tensorflow/core/kernels
for i in <op_list>
do 
    echo ====
    echo $i
    echo ====
    grep -Rl \”$i\” .
done