Question

我目前正在尝试首次在Tensorflow XLA之前进行编译工作流程，并且在尝试构建包含AOT编译对象的最终可执行二进制文件时遇到了问题。

我已经使用tutorial here生成了test_graph_tfgather.pb和test_graph_tfgather.config.pbtxt文件。然后，我直接使用tfcompile工具生成MyClass.o和MyClass.h。到目前为止一切顺利。

我现在正在构建一个包含此编译模型的简单makefile项目，但是遇到一些与Eigen相关的错误。这可能是由于在我的计算机上安装了其他版本的eigen3吗？由于本征错误，我也不得不注释掉Eigen :: ThreadPool行，因此某些版本缺少匹配可能是问题所在。有没有人以前见过这个问题，或者没有人知道如何使它工作？

谢谢。

构建错误：

g++ -c -std=c++11 -I . -I /usr/include/eigen3 -I /home/user/tensorflow_xla/tensorflow -I /usr/include main.cpp
In file included from /home/user/tensorflow_xla/tensorflow/tensorflow/compiler/xla/types.h:22:0,
                 from /home/user/tensorflow_xla/tensorflow/tensorflow/compiler/xla/executable_run_options.h:20,
                 from /home/user/tensorflow_xla/tensorflow/tensorflow/compiler/tf2xla/xla_compiled_cpu_function.h:22,
                 from MyClass.h:14,
                 from main.cpp:6:
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h: In static member function ‘static tensorflow::bfloat16 Eigen::NumTraits<tensorflow::bfloat16>::infinity()’:
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h:79:28: error: ‘infinity’ is not a member of ‘Eigen::NumTraits<float>’
     return FloatToBFloat16(NumTraits<float>::infinity());
                            ^
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h: In static member function ‘static tensorflow::bfloat16 Eigen::NumTraits<tensorflow::bfloat16>::quiet_NaN()’:
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h:83:28: error: ‘quiet_NaN’ is not a member of ‘Eigen::NumTraits<float>’
     return FloatToBFloat16(NumTraits<float>::quiet_NaN());
                            ^
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h: At global scope:
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h:95:34: error: ‘log’ is not a template function
     const tensorflow::bfloat16& x) {
                                  ^
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h:101:34: error: ‘exp’ is not a template function
     const tensorflow::bfloat16& x) {
                                  ^
/home/user/tensorflow_xla/tensorflow/tensorflow/core/framework/numeric_types.h:107:34: error: ‘abs’ is not a template function
     const tensorflow::bfloat16& x) {
                                  ^
Makefile:10: recipe for target 'main.o' failed

main.cpp来源：

#define EIGEN_USE_THREADS
#define EIGEN_USE_CUSTOM_THREAD_POOL

#include <iostream>
#include "third_party/eigen3/unsupported/Eigen/CXX11/Tensor"
#include "MyClass.h" // generated

int main(int argc, char** argv) {
  //Eigen::ThreadPool tp(2);  // Size the thread pool as appropriate.
  //Eigen::ThreadPoolDevice device(&tp, tp.NumThreads());

  MyClass matmul;
  //matmul.set_thread_pool(&device);

  // Set up args and run the computation.
  const float args[12] = {1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12};
  std::copy(args + 0, args + 6, matmul.arg0_data());
  std::copy(args + 6, args + 12, matmul.arg1_data());
  matmul.Run();

  // Check result
  if (matmul.result0(0, 0) == 58) {
    std::cout << "Success" << std::endl;
  } else {
    std::cout << "Failed. Expected value 58 at 0,0. Got:"
              << matmul.result0(0, 0) << std::endl;
  }

  return 0;
}

Makefile

EIGEN_INC=-I /usr/include/eigen3
TF_INC=-I /home/user/tensorflow_xla/tensorflow
CPPFLAGS=-c -std=c++11

xla_hw: main.o MyClass.o
    g++ -o xla_hw main.o MyClass.o

main.o: main.cpp
    g++ $(CPPFLAGS) -I . $(TF_INC) $(EIGEN_INC) -I /usr/include main.cpp

Answer 1

我现在已经解决了这个问题，事实证明，tensorflow中包含了eigen3的特定版本，您需要使用此版本才能工作。构建张量流后，正确的eigen3版本位于>>> from sklearn.feature_extraction.text import CountVectorizer >>> from sklearn.decomposition import LatentDirichletAllocation >>> from sklearn.datasets import fetch_20newsgroups >>> >>> n_samples = 2000 >>> n_features = 1000 >>> n_components = 10 >>> >>> dataset = fetch_20newsgroups(shuffle=True, random_state=1, ... remove=('headers', 'footers', 'quotes')) >>> data_samples = dataset.data[:n_samples] >>> >>> tf_vectorizer = CountVectorizer(max_df=0.95, min_df=2, ... max_features=n_features, ... stop_words='english') >>> tf = tf_vectorizer.fit_transform(data_samples) >>> >>> lda = LatentDirichletAllocation(n_components=n_components, max_iter=5, ... learning_method='online', ... learning_offset=50., ... random_state=0) >>> >>> lda.fit(tf) LatentDirichletAllocation(batch_size=128, doc_topic_prior=None, evaluate_every=-1, learning_decay=0.7, learning_method='online', learning_offset=50.0, max_doc_update_iter=100, max_iter=5, mean_change_tol=0.001, n_components=10, n_jobs=1, n_topics=None, perp_tol=0.1, random_state=0, topic_word_prior=None, total_samples=1000000.0, verbose=0) >>> >>> print(lda.transform(tf_vectorizer.transform(dataset.data[-3:]))) [[0.0142868 0.63695359 0.01428674 0.01428686 0.01428606 0.01429304 0.014286 0.24874298 0.01429136 0.01428656] [0.01111385 0.45234109 0.01111409 0.45875254 0.01111215 0.01111384 0.01111214 0.01111282 0.01111441 0.01111307] [0.001786 0.68840635 0.00178639 0.00178615 0.00178625 0.00178627 0.00178587 0.00178627 0.29730378 0.00178667]]

下面是有效的makefile，其中包括正确的Eigen路径以及链接项目所需的库。

<tensorflow path>bazel-tensorflow/external/eigen_archive

Tensorflow XLA AOT：与特征相关的错误构建项目

1 个答案: