从Source(Win7)安装Tensorflow:bazel,python - >没有名为'编码的模块

时间:2017-07-16 14:09:51

标签: python machine-learning tensorflow windows-7-x64 bazel

目标

在c / c ++中使用Python训练(keras)模型。 请参阅c-work-around的解决方法部分。

从源代码编译Tensorflow(rev 1.0或1.2.1) 按照

的说明进行操作

https://www.tensorflow.org/install/install_windows

(tensorflow已经通过python3中的pip安装了,但是我需要从c ++中使用它,所以从源代码安装似乎是强制性的)

系统

  • WIN7(64)
  • bazel 0.5.2,
  • python(env)3.5.2(64),
  • pip(env)9.0.1,
  • msys64 / msys2_shell.cmd(GNU bash,4.4.12(64-pc-msys))
  • gcc 6.3.0
  • java SE" 1.8.0_131",
  • CUDA 8.0.60(未使用)。

错误

(完整日志在下面的附加部分中)

在msys2-shell中,设置环境变量,激活python virt-env,进行bazel clean,。/ configure,

bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package

我收到此错误:

  ERROR: D:/user/sourcen/external/tensorflow-master-201707051514/
            tensorflow-master-rev-1-2-1/tensorflow/core/BUILD:1415:1: 

  Executing genrule //tensorflow/core:version_info_gen failed: 

  bash.exe failed: error executing command

  cd C:/users/user/appdata/local/temp/_bazel_user/chogfwdz/execroot/org_tensorflow

  C:/tools/msys64/usr/bin/bash.exe 
                              -c source external/bazel_tools/tools/genrule/genrule-setup.sh; 
                                        tensorflow/tools/git/gen_git_source.py 
                              --generate tensorflow/tools/git/gen/spec.json 
                                         tensorflow/tools/git/gen/head 
                                         tensorflow/tools/git/gen/branch_ref 
                                            "bazel-out/host/genfiles/tensorflow/core/util/version_info.cc": 
                                              com.google.devtools.build.lib.shell.
  BadExitStatusException: Process exited with status 3.

  Fatal Python error: Py_Initialize: unable to load the file system codec
  ImportError: No module named 'encodings'

对不起,我绝对迷失在这里,这条消息什么也没告诉我。 这里有一些上面提到的文件片段,也许有人有胶水给我一个启示的暗示....

// tensorflow /核心/ BUILD:1415:1      tf_version_info_genrule()

 cc_library(
     name = "version_lib",
     srcs = ["util/version_info.cc"],
     hdrs = ["public/version.h"],
     copts = tf_copts(),
 )

巴泽勒/.../ genrule-setup.sh

 set -e
 set -u
 set -o pipefail

// tensorflow /工具/ GIT中/ gen_git_source.py

  """... creates symlinks from the internal git repository directory so
  that the build system can see changes in the version state. ..."""

  def parse_branch_ref(filename)...
  def configure(src_base_path, debug=False)...
  def get_git_version(git_base_path)...
  def write_version_info(filename, git_version)...
  def generate(arglist)...
  def raw_generate(output_file)...

我想,使用了错误的python版本(不是我激活的virtualenv) - 还安装了其他几个python版本(2.7-32,2.7-64,3.5-32,3.5-64)(否则需要) )。

问题

如果bazel正在使用正确的python(virtualenv),我该怎么办?

如果bazel使用了错误的,我怎么能改变它?

如果bazel正在使用正确的版本,我怎样才能获得有关出错的更多细节(并修复它)?

元问题

需要哪些进一步的信息(环境变量)?

如何改善问题以使人们能够提供建议? (请发表评论,遗漏什么,错误,不清楚,太多)

后续步骤

我搜索了#34; Fatal Python错误:Py_Initialize:无法加载文件系统编解码器 ImportError:没有名为'编码的模块'"

  • 尝试使用本地管理员帐户执行以下程序 - >被公司防火墙阻止

  • 尝试设置PYTHONPATH和PYTHONHOME,没效果。

    export PYTHONHOME = / D / bin / Python64-3-Anaconda / envs / python64-35

    export PYTHONPATH = / D / bin / Python64-3-Anaconda / envs / python64-35:                   / d /斌/ Python64-3-蟒蛇/ ENVS / python64-35 / lib目录/网站套餐:                   / d / bin中/ Python64-3-阿纳康达/ ENVS / python64-35 /库

  • 尝试设置另一个virtualenv(作为管理员)

  • 尝试通过cmake安装tensorflow

程序

这是我在启动msys2后所做的:

在TF 1.0中,我针对我遇到的第一个问题(下载numeric_min_js的失败)进行了解决。 经过一些IRC谈话&很多尝试 - 最后我注释掉了一行  / tensorflow-rev1-0 / tensorflow / tensorboard / bower / BUILD(见下文)。

所以现在./configure脚本(TF 1.0)运行没有错误(关于http_proxy的警告可以被忽略,因为我已经在IRC和我用Google搜索的一些URL上告知过)。下面我添加了完整构建尝试的完整日志。

我的Python虚拟环境在

d:/bin/Python64-3-Anaconda/envs/python64-35/python.exe,

我使用的库路径:

D:/bin/Python64-3-Anaconda/envs/python64-35/lib/site-packages.

TF 1.0尝试

修补BUILD(TF1.0)

.../tensorflow-rev1-0/tensorflow/tensorboard/bower/BUILD

 filegroup(
     name = "bower",
     srcs = [
    ...
         # "@numericjs_numeric_min_js//file", # by rg20170710
    ...




 user@CRPC0815 MSYS ~     
$ cd /d/user/Sourcen/external/tensorflow-rev1-0/

步骤0:设置env,激活python-env,cd

启动msys2-shell:

 $ c:/tools/msys64/msys2_shell.cmd

设置环境变量

  user@CRPC0815 MSYS ~
 $ source d:/user/00-Projekte/VrVis/env-cuda-python64-3--1.sh

激活virtual-env

 $ cd /d/bin/Python64-3-Anaconda/
 $ source activate python64-35

转到来源

 $ cd /d/user/Sourcen/external/tensorflow-rev1-0

  (python64-35)
  user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-rev1-0
 $ 

第1步:bazel clean

 (python64-35)
 user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-rev1-0
$ bazel clean
 WARNING: ignoring http_proxy in environment.
 ....................
 INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes.

第2步:./ configure

 (python64-35)
 user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-rev1-0
$ ./configure

 Please specify the location of python.
     [Default is /D/bin/Python64-3-Anaconda/envs/python64-35/python]:

 Please specify optimization flags to use during compilation [Default is -march=native]:

 Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
  No XLA support will be enabled for TensorFlow

 Found possible Python library paths:
   D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages
   D:\bin\Python64-3-Anaconda\envs\python64-35
 Please input the desired Python library path to use.
   Default is [D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages]

 Using python library path: D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages

 Junction created for util\python\python_include <<===>> D:\bin\Python64-3-Anaconda\envs\python64-35\include

 Junction created for util\python\python_lib <<===>> D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages

 Junction created for third_party\py\numpy\numpy_include <<===>> D:\bin\Python64-3-Anaconda\envs\
                                                                    python64-35\lib\site-packages\numpy\core\include

 Do you wish to build TensorFlow with CUDA support? [y/N]
 No CUDA support will be enabled for TensorFlow

 Configuration finished

 WARNING: ignoring http_proxy in environment.
 INFO: All external dependencies fetched successfully.

第3步:bazel build&amp;错误

 $ bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package

  WARNING: ignoring http_proxy in environment.

  INFO: Found 1 target...

  ERROR: D:/user/sourcen/external/tensorflow-rev1-0/tensorflow/core/BUILD:1227:1:
   Executing genrule //tensorflow/core:version_info_gen failed:
    bash.exe failed:
     error executing command

      cd C:/users/user/appdata/local/temp/_bazel_user/hnxszhzu/execroot/org_tensorflow

      SET PATH=C:\tools\msys64\usr\bin;C:\tools\msys64\bin;
               D:\bin\Python64-3-Anaconda\envs\python64-35;
               D:\bin\Python64-3-Anaconda\envs\python64-35\Library\mingw-w64\bin;
               D:\bin\Python64-3-Anaconda\envs\python64-35\Library\usr\bin;
               D:\bin\Python64-3-Anaconda\envs\python64-35\Library\bin;
               D:\bin\Python64-3-Anaconda\envs\python64-35\Scripts;
               C:\tools\msys64\usr\bin;
               D:\user\bin\BLAS;
               D:\bin\TDM-GCC-64\bin;
               C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\bp2qlotc.p22\NativeBinaries\x86;
               C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;
               C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;
               C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64_x86\;
               C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\;
               C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\;
               C:\Program Files (x86)\MSBuild\14.0\bin;
               C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin;
               C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\libnvvp;
               C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\bin;
               C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\lib;
               C:\Program Files\TortoiseGit\bin;
               C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;
               C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
               C:\ProgramData\Oracle\Java\javapath;
               C:\Qt\Qt5.7.0\5.7\msvc2015;
               C:\WINDOWS;
               C:\WINDOWS\System32\Wbem;
               C:\WINDOWS\System32\WindowsPowerShell\v1.0\;
               C:\WINDOWS\system32;
               D:\user\bin\MinGW\msys\1.0\bin;
               D:\user\bin\MinGW\msys\1.0\sbin;
               D:\user\bin\MinGW\msys\1.0\sbin\awk;
               D:\bin\CMake\bin;
               D:\bin\Git\bin;
               [...]
               D:\bin\Python64-3-Anaconda\envs\python64-35\DLLs;
               D:\bin\Python64-3-Anaconda\envs\python64-35\Lib;
               D:\bin\Python64-3-Anaconda\envs\python64-35\libs;
               D:\bin\Slik-Subversion-1-9-2\bin;
               D:\bin\SubversionColabnet\bin;
               D:\bin\TDM-GCC-64\bin;
               C:\tools\msys64\usr\local\bin;
               C:\tools\msys64\usr\bin;
               C:\tools\msys64\usr\bin;
               C:\tools\msys64\opt\bin;
               C:\Windows\System32;
               C:\Windows;
               C:\Windows\System32\Wbem;
               C:\Windows\System32\WindowsPowerShell\v1.0\;
               C:\tools\msys64\usr\bin\site_perl;
               C:\tools\msys64\usr\bin\vendor_perl;
               C:\tools\msys64\usr\bin\core_perl

    C:/tools/msys64/usr/bin/bash.exe -c source external/bazel_tools/tools/genrule/genrule-setup.sh;
                                        tensorflow/tools/git/gen_git_source.py
                                     --generate tensorflow/tools/git/gen/spec.json
                                                tensorflow/tools/git/gen/head
                                                tensorflow/tools/git/gen/branch_ref
                                                 "bazel-out/host/genfiles/tensorflow/core/util/version_info.cc":
                                                  com.google.devtools.build.lib.

    shell.BadExitStatusException: Process exited with status 3.

  Fatal Python error: Py_Initialize: unable to load the file system codec
  ImportError: No module named 'encodings'

  Current thread 0x0000628c (most recent call first):
  Target //tensorflow/tools/pip_package:build_pip_package failed to build
  INFO: Elapsed time: 5,500s, Critical Path: 0,41s

TF 1.2.1构建尝试

步骤0:设置env,激活python-env,cd

(与TF 1.0相同)

 $ source d:/user/00-Projekte/VrVis/env-cuda-python64-3--1.sh
 $ cd /d/bin/Python64-3-Anaconda/
 $ source activate python64-35
 $ cd /d/user/Sourcen/external/tensorflow-master-201707051514/tensorflow-master-rev-1-2-1/

第1步:bazel clean

 $ bazel clean

  WARNING: ignoring http_proxy in environment.
  INFO: Starting clean (this may take a while). Consider using --async if the clean takes more than several minutes

第2步:./ configure

 $ ./configure

  WARNING: ignoring http_proxy in environment.

  You have bazel 0.5.2- installed.

  Please specify the location of python.
    [Default is /D/bin/Python64-3-Anaconda/envs/python64-35/python]:

  Found possible Python library paths:
    D:\bin\Python64-3-Anaconda\envs\python64-35
    D:\bin\Python64-3-Anaconda\envs\python64-35\lib\site-packages
  Please input the desired Python library path to use.
    Default is [D:\bin\Python64-3-Anaconda\envs\python64-35]
  D:/bin/Python64-3-Anaconda/envs/python64-35/lib/site-packages

  Do you wish to build TensorFlow with MKL support? [y/N]
  No MKL support will be enabled for TensorFlow

  Please specify optimization flags to use during compilation when bazel option "--config=opt" is specified
    [Default is -march=native]:

  Do you wish to build TensorFlow with the XLA just-in-time compiler (experimental)? [y/N]
  No XLA support will be enabled for TensorFlow

  Do you wish to build TensorFlow with VERBS support? [y/N]
  No VERBS support will be enabled for TensorFlow

  Do you wish to build TensorFlow with CUDA support? [y/N]
  No CUDA support will be enabled for TensorFlow

  Do you wish to build TensorFlow with MPI support? [y/N]
  MPI support will not be enabled for TensorFlow

  Configuration finished

第3步:bazel build&amp;错误

  (python64-35)
  user@CRPC0815 MSYS /d/user/Sourcen/external/tensorflow-master-201707051514/tensorflow-master-rev-1-2-1
 $ bazel build --verbose_failures --config=opt //tensorflow/tools/pip_package:build_pip_package

  WARNING: ignoring http_proxy in environment.

  INFO: Found 1 target...

  ERROR: D:/user/sourcen/external/tensorflow-master-201707051514/tensorflow-master-rev-1-2-1/tensorflow/core/BUILD:1415:1:
         Executing genrule //tensorflow/core:version_info_gen failed:
          bash.exe failed:
           error executing command

           cd C:/users/user/appdata/local/temp/_bazel_user/chogfwdz/execroot/org_tensorflow

           SET PATH=C:\tools\msys64\usr\bin;
                    C:\tools\msys64\bin;
                    C:\tools\msys64\usr\bin;
                    D:\user\bin\BLAS;
                    D:\bin\TDM-GCC-64\bin;
                    C:\Program Files (x86)\Microsoft Visual Studio 14.0\Common7\IDE\Extensions\bp2qlotc.p22\NativeBinaries\x86;
                    C:\Program Files (x86)\NVIDIA Corporation\PhysX\Common;
                    C:\Program Files (x86)\Windows Kits\8.1\Windows Performance Toolkit\;
                    C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\amd64_x86\;
                    C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\;
                    C:\Program Files (x86)\Microsoft Visual Studio 14.0\VC\bin\x86_amd64\;
                    C:\Program Files (x86)\MSBuild\14.0\bin;
                    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\bin;
                    C:\Program Files\NVIDIA GPU Computing Toolkit\CUDA\v8.0\libnvvp;
                    C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\bin;
                    C:\Program Files\NVIDIA GPU Computing Toolkit\cuDNN\lib;
                    C:\Program Files\TortoiseGit\bin;
                    C:\Program Files\Microsoft SQL Server\120\Tools\Binn\;
                    C:\Program Files\Microsoft SQL Server\130\Tools\Binn\;
                    C:\ProgramData\Oracle\Java\javapath;
                    C:\Qt\Qt5.7.0\5.7\msvc2015;
                    C:\WINDOWS;
                    C:\WINDOWS\System32\Wbem;
                    C:\WINDOWS\System32\WindowsPowerShell\v1.0\;
                    C:\WINDOWS\system32;
                    D:\BaslerPylon5\Runtime\Win32\;
                    D:\BaslerPylon5\Runtime\x64\;
                    D:\user\bin\MinGW\msys\1.0\bin;
                    D:\user\bin\MinGW\msys\1.0\sbin;
                    D:\user\bin\MinGW\msys\1.0\sbin\awk;
                    D:\bin\CMake\bin;
                    D:\bin\Git\bin;
                    D:\bin\Python64-3-Anaconda\envs\python64-35;
                    D:\bin\Python64-3-Anaconda\envs\python64-35\Library\bin;
                    D:\bin\Python64-3-Anaconda\envs\python64-35\Scripts;
                    D:\bin\Python64-3-Anaconda\envs\python64-35\DLLs;
                    D:\bin\Python64-3-Anaconda\envs\python64-35\Lib;
                    D:\bin\Python64-3-Anaconda\envs\python64-35\libs;
                    D:\bin\Slik-Subversion-1-9-2\bin;
                    D:\bin\SubversionColabnet\bin;
                    D:\bin\TDM-GCC-64\bin;
                    D:\bin\gnuplot\bin;
                    D:\user\bin;
    .
    C:/tools/msys64/usr/bin/bash.exe -c source external/bazel_tools/tools/genrule/genrule-setup.sh;
                                        tensorflow/tools/git/gen_git_source.py
                                     --generate tensorflow/tools/git/gen/spec.json
                                                tensorflow/tools/git/gen/head tensorflow/tools/git/gen/branch_ref
                                                 "bazel-out/host/genfiles/tensorflow/core/util/version_info.cc":
                                                  com.google.devtools.build.lib.shell.

    BadExitStatusException: Process exited with status 3.

  Fatal Python error: Py_Initialize: unable to load the file system codec
  ImportError: No module named 'encodings'

  Current thread 0x000036e8 (most recent call first):
  Target //tensorflow/tools/pip_package:build_pip_package failed to build

  INFO: Elapsed time: 5,413s, Critical Path: 0,52s

变通

与此同时,我找到了部署Tensorflow(Keras)模型(win7 / win10)的解决方法,无需使用bazel / cmake / sources ......:

  • 使用win/cpuc-api example
  • 中的C-API c_api.h
  • 通过vs2015命令提示符从dll创建lib:

    dumpbin / exports xxx.dll&gt; xxx.def

    pywrap_tensorflow_internal.lib

  • 从python中提取gpu lib pywrap_tensorflow_internal.lib

  • 将pyd lib _pywrap_tensorflow_internal.pyd复制到dll-path
  • 也许使用gpu-options(python - &gt; c):

  • python代码

    gpu_options = tensorflow.GPUOptions(allow_growth = True)

    config = tensorflow.ConfigProto(gpu_options = gpu_options)

    serialized = config.SerializeToString()

    [int(i)for i in serialized]

  • gpu-options的c代码:

    TF_SessionOptions * sess_opts = TF_NewSessionOptions();

    // tensorflow.GPUOptions(allow_growth = True)...

    uint8_t configProtoGPU [] = {50,2,32,1};

    size_t configProtoGPUlen = sizeof(configProtoGPU)/ sizeof(uint8_t);

    TF_SetConfig(sess_opts,configProtoGPU,configProtoGPUlen,status);

0 个答案:

没有答案