libgpuarray测试#10和#11失败

时间:2017-03-17 21:54:15

标签: theano lasagne theano-cuda

问题

我一直都在使用theano。与CUDA,CUDNN和CNMEM。我有一个XTITAN。 实际上我在大学服务器上运行了我的代码。

我试图安装libgpuarray,但测试#10和#11失败了。

我该怎么办?

超额信息

nvcc --version nvcc:NVIDIA(R)Cuda编译器驱动程序 版权所有(c)2005-2016 NVIDIA Corporation 建立于Tue_Jan_10_13:22:03_CST_2017 Cuda编译工具,8.0版,V8.0.61

NVIDIA-SMI

Fri Mar 17 18:35:45 2017
+-----------------------------------------------------------------------------+
| NVIDIA-SMI 367.48                 Driver Version: 367.48                    |
|-------------------------------+----------------------+----------------------+
| GPU  Name        Persistence-M| Bus-Id        Disp.A | Volatile Uncorr. ECC |
| Fan  Temp  Perf  Pwr:Usage/Cap|         Memory-Usage | GPU-Util  Compute M. |
|===============================+======================+======================|
|   0  GeForce GTX TIT...  Off  | 0000:02:00.0      On |                  N/A |
| 22%   36C    P8    15W / 250W |     69MiB / 12204MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   1  GeForce GTX TIT...  Off  | 0000:82:00.0     Off |                  N/A |
| 22%   43C    P8    16W / 250W |      1MiB / 12206MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+
|   2  GeForce GTX TIT...  Off  | 0000:83:00.0     Off |                  N/A |
| 22%   30C    P8    14W / 250W |      1MiB / 12206MiB |      0%      Default |
+-------------------------------+----------------------+----------------------+

+-----------------------------------------------------------------------------+
| Processes:                                                       GPU Memory |
|  GPU       PID  Type  Process name                               Usage      |
|=============================================================================|
|    0      1437    G   /usr/bin/Xorg                                   40MiB |
|    0      3011    G   gnome-shell                                     27MiB |
+-----------------------------------------------------------------------------+

如何安装libgpuarray

rm -rf build Build
mkdir Build
cd Build
cmake .. -DCMAKE_INSTALL_PREFIX=~/.local -DCMAKE_BUILD_TYPE=Release
make
make install

...

(myVE) andromeda@REDACTED:~/private/libgpuarray/Build$ cmake .. -DCMAKE_INSTALL_PREFIX=~/.local -DCMAKE_BUILD_TYPE=Release
-- The C compiler identification is GNU 4.9.2
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Looking for strlcat
-- Looking for strlcat - not found
-- Looking for mkstemp
-- Looking for mkstemp - found
-- Found PkgConfig: /usr/bin/pkg-config (found version "0.28")
-- checking for one of the modules 'check'
-- Looking for ck_assert_ptr_ne
-- Looking for ck_assert_ptr_ne - found
-- Found MPI_C: /usr/lib/libmpi.so;/usr/lib/x86_64-linux-gnu/libdl.so;/usr/lib/x86_64-linux-gnu/libhwloc.so
-- Configuring done
-- Generating done
-- Build files have been written to: /home/andromeda/private/libgpuarray/Build





(myVE) andromeda@REDACTED:~/private/libgpuarray/Build$ make

Scanning dependencies of target gpuarray
[  1%] Building C object src/CMakeFiles/gpuarray.dir/cache/lru.c.o
[  2%] Building C object src/CMakeFiles/gpuarray.dir/cache/twoq.c.o
[  3%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_types.c.o
[  4%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_error.c.o
[  5%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_util.c.o
[  6%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_buffer.c.o
[  7%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_buffer_blas.c.o
[  8%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_buffer_collectives.c.o
[  9%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_array.c.o
[ 10%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_array_blas.c.o
[ 11%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_array_collectives.c.o
[ 12%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_kernel.c.o
[ 13%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_extension.c.o
[ 14%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_elemwise.c.o
[ 15%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_reduction.c.o
[ 16%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_buffer_cuda.c.o
[ 17%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_blas_cuda_cublas.c.o
[ 18%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_collectives_cuda_nccl.c.o
[ 19%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_buffer_opencl.c.o
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c: In function ‘cl_free_ctx.part.5’:
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c:223:15: warning: ‘blas_ops’ may be used uninitialized in this function [-Wmaybe-uninitialized]
       blas_ops->teardown((gpucontext *)ctx);
               ^
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c: In function ‘cl_deinit’:
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c:223:15: warning: ‘blas_ops’ may be used uninitialized in this function [-Wmaybe-uninitialized]
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c:215:22: note: ‘blas_ops’ was declared here
   gpuarray_blas_ops *blas_ops;
                      ^
[ 20%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_blas_opencl_clblas.c.o
[ 21%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_blas_opencl_clblast.c.o
[ 22%] Building C object src/CMakeFiles/gpuarray.dir/gpuarray_strl.c.o
[ 23%] Building C object src/CMakeFiles/gpuarray.dir/util/strb.c.o
[ 24%] Building C object src/CMakeFiles/gpuarray.dir/util/xxhash.c.o
[ 25%] Building C object src/CMakeFiles/gpuarray.dir/util/integerfactoring.c.o
[ 26%] Building C object src/CMakeFiles/gpuarray.dir/loaders/dyn_load.c.o
[ 27%] Building C object src/CMakeFiles/gpuarray.dir/loaders/libcuda.c.o
/home/andromeda/private/libgpuarray/src/loaders/libcuda.c: In function ‘load_libcuda’:
/home/andromeda/private/libgpuarray/src/loaders/libcuda.c:46:9: warning: unused variable ‘v’ [-Wunused-variable]
   float v;
         ^
[ 28%] Building C object src/CMakeFiles/gpuarray.dir/loaders/libnvrtc.c.o
[ 29%] Building C object src/CMakeFiles/gpuarray.dir/loaders/libcublas.c.o
[ 30%] Building C object src/CMakeFiles/gpuarray.dir/loaders/libnccl.c.o
[ 31%] Building C object src/CMakeFiles/gpuarray.dir/loaders/libopencl.c.o
[ 32%] Building C object src/CMakeFiles/gpuarray.dir/loaders/libclblas.c.o
[ 33%] Building C object src/CMakeFiles/gpuarray.dir/loaders/libclblast.c.o
Linking C shared library ../../lib/libgpuarray.so
[ 34%] Built target gpuarray
Scanning dependencies of target gpuarray-static
[ 35%] Building C object src/CMakeFiles/gpuarray-static.dir/cache/lru.c.o
[ 36%] Building C object src/CMakeFiles/gpuarray-static.dir/cache/twoq.c.o
[ 37%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_types.c.o
[ 38%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_error.c.o
[ 39%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_util.c.o
[ 40%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_buffer.c.o
[ 41%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_buffer_blas.c.o
[ 42%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_buffer_collectives.c.o
[ 43%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_array.c.o
[ 44%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_array_blas.c.o
[ 45%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_array_collectives.c.o
[ 46%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_kernel.c.o
[ 47%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_extension.c.o
[ 48%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_elemwise.c.o
[ 49%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_reduction.c.o
[ 50%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_buffer_cuda.c.o
[ 51%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_blas_cuda_cublas.c.o
[ 52%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_collectives_cuda_nccl.c.o
[ 53%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_buffer_opencl.c.o
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c: In function ‘cl_free_ctx.part.5’:
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c:223:15: warning: ‘blas_ops’ may be used uninitialized in this function [-Wmaybe-uninitialized]
       blas_ops->teardown((gpucontext *)ctx);
               ^
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c: In function ‘cl_deinit’:
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c:223:15: warning: ‘blas_ops’ may be used uninitialized in this function [-Wmaybe-uninitialized]
/home/andromeda/private/libgpuarray/src/gpuarray_buffer_opencl.c:215:22: note: ‘blas_ops’ was declared here
   gpuarray_blas_ops *blas_ops;
                      ^
[ 54%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_blas_opencl_clblas.c.o
[ 55%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_blas_opencl_clblast.c.o
[ 56%] Building C object src/CMakeFiles/gpuarray-static.dir/gpuarray_strl.c.o
[ 57%] Building C object src/CMakeFiles/gpuarray-static.dir/util/strb.c.o
[ 58%] Building C object src/CMakeFiles/gpuarray-static.dir/util/xxhash.c.o
[ 59%] Building C object src/CMakeFiles/gpuarray-static.dir/util/integerfactoring.c.o
[ 60%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/dyn_load.c.o
[ 61%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/libcuda.c.o
/home/andromeda/private/libgpuarray/src/loaders/libcuda.c: In function ‘load_libcuda’:
/home/andromeda/private/libgpuarray/src/loaders/libcuda.c:46:9: warning: unused variable ‘v’ [-Wunused-variable]
   float v;
         ^
[ 62%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/libnvrtc.c.o
[ 63%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/libcublas.c.o
[ 64%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/libnccl.c.o
[ 65%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/libopencl.c.o
[ 66%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/libclblas.c.o
[ 67%] Building C object src/CMakeFiles/gpuarray-static.dir/loaders/libclblast.c.o
Linking C static library ../../lib/libgpuarray-static.a
[ 68%] Built target gpuarray-static
Scanning dependencies of target check_array
[ 69%] Building C object tests/CMakeFiles/check_array.dir/main.c.o
[ 70%] Building C object tests/CMakeFiles/check_array.dir/device.c.o
[ 71%] Building C object tests/CMakeFiles/check_array.dir/check_array.c.o
Linking C executable check_array
[ 71%] Built target check_array
Scanning dependencies of target check_blas
[ 72%] Building C object tests/CMakeFiles/check_blas.dir/main.c.o
[ 73%] Building C object tests/CMakeFiles/check_blas.dir/device.c.o
[ 74%] Building C object tests/CMakeFiles/check_blas.dir/check_blas.c.o
Linking C executable check_blas
[ 74%] Built target check_blas
Scanning dependencies of target check_buffer
[ 75%] Building C object tests/CMakeFiles/check_buffer.dir/main.c.o
[ 76%] Building C object tests/CMakeFiles/check_buffer.dir/device.c.o
[ 77%] Building C object tests/CMakeFiles/check_buffer.dir/check_buffer.c.o
Linking C executable check_buffer
[ 77%] Built target check_buffer
Scanning dependencies of target check_buffer_collectives
[ 78%] Building C object tests/CMakeFiles/check_buffer_collectives.dir/main.c.o
[ 79%] Building C object tests/CMakeFiles/check_buffer_collectives.dir/device.c.o
[ 80%] Building C object tests/CMakeFiles/check_buffer_collectives.dir/communicator.c.o
[ 81%] Building C object tests/CMakeFiles/check_buffer_collectives.dir/check_buffer_collectives.c.o
Linking C executable check_buffer_collectives
[ 81%] Built target check_buffer_collectives
Scanning dependencies of target check_collectives
[ 82%] Building C object tests/CMakeFiles/check_collectives.dir/main.c.o
[ 83%] Building C object tests/CMakeFiles/check_collectives.dir/device.c.o
[ 84%] Building C object tests/CMakeFiles/check_collectives.dir/communicator.c.o
[ 85%] Building C object tests/CMakeFiles/check_collectives.dir/check_collectives.c.o
Linking C executable check_collectives
[ 85%] Built target check_collectives
Scanning dependencies of target check_elemwise
[ 86%] Building C object tests/CMakeFiles/check_elemwise.dir/main.c.o
[ 87%] Building C object tests/CMakeFiles/check_elemwise.dir/device.c.o
[ 88%] Building C object tests/CMakeFiles/check_elemwise.dir/check_elemwise.c.o
Linking C executable check_elemwise
[ 88%] Built target check_elemwise
Scanning dependencies of target check_error
[ 89%] Building C object tests/CMakeFiles/check_error.dir/main.c.o
[ 90%] Building C object tests/CMakeFiles/check_error.dir/check_error.c.o
Linking C executable check_error
[ 90%] Built target check_error
Scanning dependencies of target check_reduction
[ 91%] Building C object tests/CMakeFiles/check_reduction.dir/main.c.o
[ 92%] Building C object tests/CMakeFiles/check_reduction.dir/device.c.o
[ 93%] Building C object tests/CMakeFiles/check_reduction.dir/check_reduction.c.o
Linking C executable check_reduction
[ 93%] Built target check_reduction
Scanning dependencies of target check_types
[ 94%] Building C object tests/CMakeFiles/check_types.dir/main.c.o
[ 95%] Building C object tests/CMakeFiles/check_types.dir/check_types.c.o
Linking C executable check_types
[ 95%] Built target check_types
Scanning dependencies of target check_util
[ 96%] Building C object tests/CMakeFiles/check_util.dir/main.c.o
[ 97%] Building C object tests/CMakeFiles/check_util.dir/check_util.c.o
Linking C executable check_util
[ 97%] Built target check_util
Scanning dependencies of target check_util_integerfactoring
[ 98%] Building C object tests/CMakeFiles/check_util_integerfactoring.dir/main.c.o
[100%] Building C object tests/CMakeFiles/check_util_integerfactoring.dir/check_util_integerfactoring.c.o
Linking C executable check_util_integerfactoring
[100%] Built target check_util_integerfactoring

------------------ 然后:

(myVE) andromeda@REDACTED:~/private/libgpuarray/Build$ make install
[ 34%] Built target gpuarray
[ 68%] Built target gpuarray-static
[ 71%] Built target check_array
[ 74%] Built target check_blas
[ 77%] Built target check_buffer
[ 81%] Built target check_buffer_collectives
[ 85%] Built target check_collectives
[ 88%] Built target check_elemwise
[ 90%] Built target check_error
[ 93%] Built target check_reduction
[ 95%] Built target check_types
[ 97%] Built target check_util
[100%] Built target check_util_integerfactoring
Install the project...
-- Install configuration: "Release"
-- Installing: /home/andromeda/.local/include/gpuarray/array.h
-- Installing: /home/andromeda/.local/include/gpuarray/blas.h
-- Installing: /home/andromeda/.local/include/gpuarray/collectives.h
-- Installing: /home/andromeda/.local/include/gpuarray/buffer.h
-- Installing: /home/andromeda/.local/include/gpuarray/buffer_blas.h
-- Installing: /home/andromeda/.local/include/gpuarray/buffer_collectives.h
-- Installing: /home/andromeda/.local/include/gpuarray/abi_version.h
-- Installing: /home/andromeda/.local/include/gpuarray/config.h
-- Installing: /home/andromeda/.local/include/gpuarray/elemwise.h
-- Installing: /home/andromeda/.local/include/gpuarray/error.h
-- Installing: /home/andromeda/.local/include/gpuarray/extension.h
-- Installing: /home/andromeda/.local/include/gpuarray/ext_cuda.h
-- Installing: /home/andromeda/.local/include/gpuarray/kernel.h
-- Installing: /home/andromeda/.local/include/gpuarray/types.h
-- Installing: /home/andromeda/.local/include/gpuarray/util.h
-- Installing: /home/andromeda/.local/lib/libgpuarray.so.2.0
-- Up-to-date: /home/andromeda/.local/lib/libgpuarray.so.2
-- Up-to-date: /home/andromeda/.local/lib/libgpuarray.so
-- Installing: /home/andromeda/.local/lib/libgpuarray-static.a

测试:

(myVE) andromeda@REDACTED:~/private/libgpuarray/Build$ DEVICE=cuda2 make test
Running tests...
Test project /home/andromeda/private/libgpuarray/Build
      Start  1: test_types
 1/11 Test  #1: test_types .......................   Passed    0.01 sec
      Start  2: test_util
 2/11 Test  #2: test_util ........................   Passed    0.01 sec
      Start  3: test_util_integerfactoring
 3/11 Test  #3: test_util_integerfactoring .......   Passed    0.67 sec
      Start  4: test_reduction
 4/11 Test  #4: test_reduction ...................   Passed    8.30 sec
      Start  5: test_array
 5/11 Test  #5: test_array .......................   Passed    3.13 sec
      Start  6: test_blas
 6/11 Test  #6: test_blas ........................   Passed    3.49 sec
      Start  7: test_elemwise
 7/11 Test  #7: test_elemwise ....................   Passed   25.22 sec
      Start  8: test_error
 8/11 Test  #8: test_error .......................   Passed    0.02 sec
      Start  9: test_buffer
 9/11 Test  #9: test_buffer ......................   Passed    4.77 sec
      Start 10: test_buffer_collectives
10/11 Test #10: test_buffer_collectives ..........***Failed    0.87 sec
      Start 11: test_collectives
11/11 Test #11: test_collectives .................***Failed    0.85 sec

82% tests passed, 2 tests failed out of 11

Total Test time (real) =  47.37 sec

The following tests FAILED:
         10 - test_buffer_collectives (Failed)
         11 - test_collectives (Failed)
Errors while running CTest
Makefile:117: recipe for target 'test' failed
make: *** [test] Error 8
(myVE) andromeda@REDACTED:~/private/libgpuarray/Build$

1 个答案:

答案 0 :(得分:0)

根据github libgpuarray问题:

  

最后两个测试需要nccl,如果不存在则会失败。如果您不尝试使用nccl,则可以忽略这些失败。

-

https://github.com/Theano/libgpuarray/issues/383#issuecomment-287491789