我可以使用Anaconda提供的MKL库和Spark

Question

我想使用随Anaconda提供的MKL库和Spark。我安装了libgfortran

conda install libgfortran

链接库

ln -s $PATH_TO_ANACONDA/lib/libmkl_rt.so  $PATH_TO_ANACONDA/lib/liblapack.so.3
ln -s  $PATH_TO_ANACONDA/lib/libmkl_rt.so $ PATH_TO_ANACONDA/lib/libblas.so.3 

并在LD_LIBRARY_PATH上放置PATH_TO_ANACONDA / lib

export LD_LIBRARY_PATH=$LD_LIBRARY_PATH:$PATH_TO_ANACONDA/lib

但是当我运行Spark ALS example时，我看到了

WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeSystemBLAS
WARN BLAS: Failed to load implementation from: com.github.fommil.netlib.NativeRefBLAS
WARN LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeSystemLAPACK
WARN LAPACK: Failed to load implementation from: com.github.fommil.netlib.NativeRefLAPACK

所以我认为它不起作用。这有什么明显的错误吗？

Answer 1

只要将其加载到节点，就可以使用任何库。将--packages与您想要使用的库一起使用