CRAN R包的conda构建失败

时间:2018-03-30 10:49:37

标签: r anaconda conda cran miniconda

我正在尝试使用此处显示的appra从CRAN包中构建一个conda-forge上不存在的R conda包:https://www.anaconda.com/blog/developer-blog/conda-data-science/

我使用R包rann作为示例 - 这个包已经在conda-forge上以r-rann的形式存在,但我用它来测试构建过程是否正常工作。

在构建过程中,执行包含行$R CMD INSTALL --build .的脚本“conda_build.sh”。我的问题是构建过程失败,错误Fatal error: cannot open file 'CMD': No such file or directory独立于我尝试构建的包。我不知道环境变量(?)“CMD”应该来自哪里。

我采取了以下步骤:

  1. conda skeleton cran rann
  2. conda build r-rann
  3. 我还测试过从pip构建Python包,这些工作没有任何问题,所以看起来这不是我的Miniconda安装的一般问题。

    我的系统:

    • Ubuntu 17.10
    • Miniconda 3,版本4.4.10

    以下是构建过程的完整日志:

    eega@eega-nb:~/workspace$ conda build r-rann/
    Adding in variants from internal_defaults
    INFO:conda_build.variants:Adding in variants from internal_defaults
    Attempting to finalize metadata for r-rann
    INFO:conda_build.metadata:Attempting to finalize metadata for r-rann
    Solving environment: ...working... done
    
    
    ==> WARNING: A newer version of conda exists. <==
      current version: 4.4.10
      latest version: 4.5.0
    
    Please update conda by running
    
        $ conda update -n base conda
    
    
    Solving environment: ...working... done
    
    
    ==> WARNING: A newer version of conda exists. <==
      current version: 4.4.10
      latest version: 4.5.0
    
    Please update conda by running
    
        $ conda update -n base conda
    
    
    Solving environment: ...working... done
    
    
    ==> WARNING: A newer version of conda exists. <==
      current version: 4.4.10
      latest version: 4.5.0
    
    Please update conda by running
    
        $ conda update -n base conda
    
    
    BUILD START: ['r-rann-2.5.1-r341h599a50d_0.tar.bz2']
    Solving environment: ...working... done
    
    
    ==> WARNING: A newer version of conda exists. <==
      current version: 4.4.10
      latest version: 4.5.0
    
    Please update conda by running
    
        $ conda update -n base conda
    
    
    
    ## Package Plan ##
    
      environment location: /tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold
    
    
    The following NEW packages will be INSTALLED:
    
        bzip2:           1.0.6-1          conda-forge
        ca-certificates: 2018.1.18-0      conda-forge
        cairo:           1.14.10-0        conda-forge
        curl:            7.59.0-0         conda-forge
        fontconfig:      2.12.6-0         conda-forge
        freetype:        2.8.1-0          conda-forge
        gettext:         0.19.8.1-0       conda-forge
        glib:            2.55.0-0         conda-forge
        graphite2:       1.3.11-0         conda-forge
        gsl:             2.1-2            conda-forge
        harfbuzz:        1.7.6-0          conda-forge
        icu:             58.2-0           conda-forge
        jpeg:            9b-2             conda-forge
        krb5:            1.14.2-0         conda-forge
        libffi:          3.2.1-3          conda-forge
        libgcc:          7.2.0-h69d50b8_2            
        libgcc-ng:       7.2.0-hdf63c60_3            
        libiconv:        1.15-0           conda-forge
        libpng:          1.6.34-0         conda-forge
        libssh2:         1.8.0-2          conda-forge
        libstdcxx-ng:    7.2.0-hdf63c60_3            
        libtiff:         4.0.9-0          conda-forge
        libxml2:         2.9.8-0          conda-forge
        ncurses:         5.9-10           conda-forge
        openssl:         1.0.2n-0         conda-forge
        pango:           1.40.14-0        conda-forge
        pcre:            8.39-0           conda-forge
        pixman:          0.34.0-1         conda-forge
        r-base:          3.4.1-3          conda-forge
        readline:        7.0-0            conda-forge
        tk:              8.6.7-0          conda-forge
        xz:              5.2.3-0          conda-forge
        zlib:            1.2.11-0         conda-forge
    
    Preparing transaction: ...working... done
    Verifying transaction: ...working... done
    Executing transaction: ...working... done
    Solving environment: ...working... done
    
    
    ==> WARNING: A newer version of conda exists. <==
      current version: 4.4.10
      latest version: 4.5.0
    
    Please update conda by running
    
        $ conda update -n base conda
    
    
    Solving environment: ...working... done
    
    
    ==> WARNING: A newer version of conda exists. <==
      current version: 4.4.10
      latest version: 4.5.0
    
    Please update conda by running
    
        $ conda update -n base conda
    
    
    
    ## Package Plan ##
    
      environment location: /tmp/conda-builds/r-rann_1522400915206/_build_env
    
    
    The following NEW packages will be INSTALLED:
    
        binutils_impl_linux-64: 2.28.1-had2808c_3            
        binutils_linux-64:      7.2.0-26                     
        gcc_impl_linux-64:      7.2.0-habb00fd_3             
        gcc_linux-64:           7.2.0-26                     
        gxx_impl_linux-64:      7.2.0-hdf63c60_3             
        gxx_linux-64:           7.2.0-26                     
        libstdcxx-ng:           7.2.0-hdf63c60_3             
        make:                   4.2.1-0           conda-forge
    
    Preparing transaction: ...working... done
    Verifying transaction: ...working... done
    Executing transaction: ...working... done
    Source cache directory is: /tmp/conda-builds/src_cache
    Found source in cache: RANN_2.5.1_75277e5d8a.tar.gz
    Extracting download
    source tree in: /tmp/conda-builds/r-rann_1522400915206/work
    INFO: activate-binutils_linux-64.sh made the following environmental changes:
    +ADDR2LINE=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-addr2line
    +AR=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ar
    +AS=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-as
    +CXXFILT=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++filt
    +ELFEDIT=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-elfedit
    +GPROF=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gprof
    +HOST=x86_64-conda_cos6-linux-gnu
    +LD_GOLD=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ld.gold
    +LD=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ld
    +NM=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-nm
    +OBJCOPY=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-objcopy
    +OBJDUMP=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-objdump
    +RANLIB=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-ranlib
    +READELF=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-readelf
    +SIZE=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-size
    +STRINGS=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-strings
    +STRIP=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-strip
    INFO: activate-gcc_linux-64.sh made the following environmental changes:
    +CC=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-cc
    +CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
    +_CONDA_PYTHON_SYSCONFIGDATA_NAME=_sysconfigdata_x86_64_conda_cos6_linux_gnu
    +CPPFLAGS=-DNDEBUG -D_FORTIFY_SOURCE=2 -O2
    +CPP=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-cpp
    +DEBUG_CFLAGS=-march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
    +DEBUG_CPPFLAGS=-D_DEBUG -D_FORTIFY_SOURCE=2 -Og
    +GCC_AR=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-ar
    +GCC_NM=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-nm
    +GCC_RANLIB=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc-ranlib
    +GCC=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-gcc
    +LDFLAGS=-Wl,-O2 -Wl,--sort-common -Wl,--as-needed -Wl,-z,relro -Wl,-z,now -Wl,-rpath,/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib -L/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/lib
    INFO: activate-gxx_linux-64.sh made the following environmental changes:
    +CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-strong -fno-plt -O2 -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
    +CXX=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-c++
    +DEBUG_CXXFLAGS=-fvisibility-inlines-hidden -std=c++17 -fmessage-length=0 -march=nocona -mtune=haswell -ftree-vectorize -fPIC -fstack-protector-all -fno-plt -Og -g -Wall -Wextra -fvar-tracking-assignments -pipe -I/tmp/conda-builds/r-rann_1522400915206/_h_env_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold_placehold/include -fdebug-prefix-map=${SRC_DIR}=/usr/local/src/conda/${PKG_NAME}-${PKG_VERSION} -fdebug-prefix-map=${PREFIX}=/usr/local/src/conda-prefix
    +GXX=/tmp/conda-builds/r-rann_1522400915206/_build_env/bin/x86_64-conda_cos6-linux-gnu-g++
    Fatal error: cannot open file 'CMD': No such file or directory
    Traceback (most recent call last):
      File "/home/eega/miniconda3/bin/conda-build", line 6, in <module>
        sys.exit(conda_build.cli.main_build.main())
      File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 420, in main
        execute(sys.argv[1:])
      File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/cli/main_build.py", line 411, in execute
        verify=args.verify)
      File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/api.py", line 199, in build
        notest=notest, need_source_download=need_source_download, variants=variants)
      File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/build.py", line 2097, in build_tree
        notest=notest,
      File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/build.py", line 1370, in build
        utils.check_call_env(cmd, env=env, cwd=src_dir, stats=build_stats)
      File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/utils.py", line 301, in check_call_env
        return _func_defaulting_env_to_os_environ('call', *popenargs, **kwargs)
      File "/home/eega/miniconda3/lib/python3.6/site-packages/conda_build/utils.py", line 280, in _func_defaulting_env_to_os_environ
        raise subprocess.CalledProcessError(proc.returncode, _args)
    subprocess.CalledProcessError: Command '['/bin/bash', '-e', '/tmp/conda-builds/r-rann_1522400915206/work/conda_build.sh']' returned non-zero exit status 2.
    

    build.sh conda的内容已创建:

    #!/bin/bash
    
    if [[ $target_platform =~ linux.* ]] || [[ $target_platform == win-32 ]] || [[ $target_platform == win-64 ]] || [[ $target_platform == osx-64 ]]; then
      export DISABLE_AUTOBREW=1
      mv DESCRIPTION DESCRIPTION.old
      grep -v '^Priority: ' DESCRIPTION.old > DESCRIPTION
      $R CMD INSTALL --build .
    else
      mkdir -p $PREFIX/lib/R/library/RANN
      mv * $PREFIX/lib/R/library/RANN
    fi
    

    meta.yaml conda的内容创建:

    {% set version = '2.5.1' %}
    
    {% set posix = 'm2-' if win else '' %}
    {% set native = 'm2w64-' if win else '' %}
    
    package:
      name: r-rann
      version: {{ version|replace("-", "_") }}
    
    source:
      fn: RANN_2.5.1.tar.gz
      url:
        - {{ cran_mirror }}/src/contrib/RANN_2.5.1.tar.gz
        - {{ cran_mirror }}/src/contrib/Archive/RANN/RANN_2.5.1.tar.gz
      sha256: 75277e5d8a13ca01ff387f99d403268a8077862d4e95b076b74fb1b5538a8546
    
    build:
      merge_build_host: True  # [win]
      # If this is a new build for the same version, increment the build number.
      number: 0
    
      # This is required to make R link correctly on Linux.
      rpaths:
        - lib/R/lib/
        - lib/
    
    # Suggests: testthat
    requirements:
      build:
        - {{ compiler('c') }}          # [not win]
        - {{ compiler('cxx') }}        # [not win]
        - {{native}}toolchain          # [win]
        - {{posix}}filesystem          # [win]
        - {{posix}}make
    
      host:
        - r-base
    
      run:
        - r-base
        - {{native}}gcc-libs           # [win]
    
    test:
      commands:
        # You can put additional test commands to be run here.
        - $R -e "library('RANN')"           # [not win]
        - "\"%R%\" -e \"library('RANN')\""  # [win]
    
      # You can also put a file called run_test.py, run_test.sh, or run_test.bat
      # in the recipe that will be run at test time.
    
      # requires:
        # Put any additional test requirements here.
    
    about:
      home: https://github.com/jefferis/RANN
      license: GPL (>= 3)
      summary: Finds the k nearest neighbours for every point in a given dataset in O(N log N) time
        using Arya and Mount's ANN library (v1.1.3). There is support for approximate as
        well as exact searches, fixed radius searches and 'bd' as well as 'kd' trees. The
        distance is computed using the L2 (Euclidean) metric. Please see package 'RANN.L1'
        for the same functionality using the L1 (Manhattan, taxicab) metric.
      license_family: GPL3
    
    # The original CRAN metadata for this package was:
    
    # Package: RANN
    # Title: Fast Nearest Neighbour Search (Wraps ANN Library) Using L2 Metric
    # Author: Sunil Arya and David Mount (for ANN), Samuel E. Kemp, Gregory Jefferis
    # Maintainer: Gregory Jefferis <jefferis@gmail.com>
    # Copyright: ANN library is copyright University of Maryland and Sunil Arya and David Mount. See file COPYRIGHT for details.
    # Description: Finds the k nearest neighbours for every point in a given dataset in O(N log N) time using Arya and Mount's ANN library (v1.1.3). There is support for approximate as well as exact searches, fixed radius searches and 'bd' as well as 'kd' trees. The distance is computed using the L2 (Euclidean) metric. Please see package 'RANN.L1' for the same functionality using the L1 (Manhattan, taxicab) metric.
    # URL: https://github.com/jefferis/RANN
    # BugReports: https://github.com/jefferis/RANN/issues
    # Encoding: UTF-8
    # License: GPL (>= 3)
    # Suggests: testthat
    # Version: 2.5.1
    # RoxygenNote: 6.0.1
    # NeedsCompilation: yes
    # Packaged: 2017-05-19 13:52:30 UTC; jefferis
    # Repository: CRAN
    # Date/Publication: 2017-05-21 07:56:50 UTC
    
    # See
    # http://docs.continuum.io/conda/build.html for
    # more information about meta.yaml
    

    任何帮助将不胜感激!

1 个答案:

答案 0 :(得分:1)

&#34; CMD&#34;是一个R命令行指令。在conda-build期间,您下载RANN的源代码,然后在临时环境中使用R构建一个包,然后安装它(暂时)以检查它是否有效。

该行只是告诉R构建包rann。在您开发R软件包时,可以使用R CMD build <my_package>R CMD INSTALL --build <my_package>来创建该软件包的可安装存档。

看起来你的conda-build无法找到构建软件包的R,尽管这通常是在运行conda-build时设置的临时环境中完成的;并且构建过程几乎肯定会安装r-base所以没有真正的理由不能找到R。

我使用conda-build 3.7.2和conda 4.3.34以及R v3.4.1(详细输出)运行windows-subsystem for linux(有效Ubuntu 16.04)下的代码

conda skeleton cran rann
conda build r-rann --R=3.4.1 --debug

它尝试在我的系统上运行Rscript CMD INSTALL --build,而不是R CMD build。前者并不是如何在R下构建软件包,而在此步骤中,&#34;无法找到文件CMD&#34;错误被抛出,所以我更详细地研究了它。

首先,我将meta.yaml与conda-forge上的meta.yaml进行了比较。 libgcc缺少运行要求,并且r-base不存在于conda骨架设置的yaml中的构建要求中 - 我添加了这些并重建(尽管我怀疑这些导致问题)。这并没有解决任何问题,&#34;无法找到CMD&#34;错误仍然存​​在。

我在conda-build检查了更改日志。 Rscript用于从3.7.2版开始运行测试而不是R。我恢复了conda-build = 3.7.1:

conda install conda-build=3.7.1 -vv
conda build r-rann --R=3.4.1 --debug

现在,我通过ccall到R CMD INSTALL --build ...,但在编译期间仍然出现错误;所以我还没有解决问题。

尽管如此,看起来您最初发布的错误源于对conda-build的更改(使用Rscript而不是 在测试软件包时R导致了构建R软件包的问题。你能不能将这个问题交叉引用到conda-build github上。

&lt; - 编辑 - &gt;

经过一番搜索,我得到了这个工作:a)安装gfortran和b)使用conda-build 3.6.0创建骨架并构建包。 3.7.5 / 1的conda-build抛出了与版本号相关的错误(没有--version XXX参数,conda-skeleton引发了一个错误,名称空间没有版本;使用--version XXX参数,conda-skeleton引发没有为R包实现错误版本号。)

以上错误已在conda-build 3.8.0中修复,但我尚未使用