如何编译AMBER14(Tesla K40c加速版)

如何编译AMBER14(Tesla K40c加速版)

一、编译环境与要解决的问题

操作系统：RHEL6
GPU: NVIDIA Tesla K40c
AMBER: 14
硬件：HP Z820

编译出现的问题：AMBER14不支持使用CUDA7.0, 仅支持6.5或更低版本；RHEL6操作系统安装CUDA6.5不识别Tesla K40c， 需要CUDA7.0。

二、解决方式

安装CUDA7.0做为驱动，安装CUDA6.5作为runtime。

三、具体安装方式

1, 安装第三方资源库：epel-release-6-8.noarch.rpm

wget http://mirrors.opencas.cn/epel/6/x86_64/epel-release-6-8.noarch.rpm

rpm -i http://mirrors.opencas.cn/epel/6/x86_64/epel-release-6-8.noarch.rpm

这个资源库注释有问题，需要进行修改
vim /etc/yum.repos.d/epel.repo
修改下面两行
#baseurl=xxxx
mirrorlist=https://####
为：
baseurl=xxxx
#mirrorlist=xxxx

＊如果不修改，有可能yum install cuda回提示epel meta link错误

2，安装CUDA 7

CUDA7作为TESLA K40c的驱动

1）从nvidia网站上下载，CUDA7的Repository文件，rpm安装
rpm –install cuda-repo-rhel6-7-0-28.x86_64.rpm

2）清除过期meta－data
yum clean expire-cache

3）安装cuda 7
yum install cuda

注：如果之前没有修改第三方资源库epel的话，会出现epel的问题而没有办法继续安装

成功后，CUDA的目录为：
/usr/local/cuda

编译AMBER，需要声明CUDA_HOME变量：
export CUDA_HOME=/usr/local/cuda

4）尝试安装AMBER14

安装amber
./configure -cuda gnu
make install

＊出现错误：cuda版本太高，只能用6.5以下的
Error: Unsupported CUDA version 7.0 detected.

AMBER requires CUDA version == 5.0 .or. 5.5 .or. 6.0 .or. 6.5

Configure failed due to the errors above!

3.安装CUDA6.5,提供Runtime

1）删除/usr/local/cuda目录

rm -fr /usr/local/cuda

这个目录将被CUDA6.5的内容覆盖

2）卸载cuda-repo-rhel6-7-0-28.x86_64.rpm

rpm -e cuda-repo-rhel6-7xxxx #xxxx代表具体的版本号

3）安装 CUDA 6.5
按照上面步骤2. 安装CUDA 7的流程，从下载 cuda-repo-rhel6-6.5-14.x86_64.rpm开始，安装CUDA6.5

4, 驱动测试

CUDA自带有examples，编译进行测试。比如：
设备验证
cp /usr/local/cuda/samples .
cd samples
cd 1_Utilities
make
./deviceQuery
[amber@amtech deviceQuery]$ ./deviceQuery
./deviceQuery Starting…
——————————————————————————————–
CUDA Device Query (Runtime API) version (CUDART static linking)

Detected 2 CUDA Capable device(s)

Device 0: “Tesla K40c”
CUDA Driver Version / Runtime Version 7.0 / 6.5
CUDA Capability Major/Minor version number: 3.5
Total amount of global memory: 11520 MBytes (12079136768 bytes)
(15) Multiprocessors, (192) CUDA Cores/MP: 2880 CUDA Cores
GPU Clock rate: 745 MHz (0.75 GHz)
Memory Clock rate: 3004 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 1572864 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Enabled
Device supports Unified Addressing (UVA): Yes
Device PCI Bus ID / PCI location ID: 6 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >

Device 1: “Tesla K40c”
CUDA Driver Version / Runtime Version 7.0 / 6.5
CUDA Capability Major/Minor version number: 3.5
Total amount of global memory: 11520 MBytes (12079136768 bytes)
(15) Multiprocessors, (192) CUDA Cores/MP: 2880 CUDA Cores
GPU Clock rate: 745 MHz (0.75 GHz)
Memory Clock rate: 3004 Mhz
Memory Bus Width: 384-bit
L2 Cache Size: 1572864 bytes
Maximum Texture Dimension Size (x,y,z) 1D=(65536), 2D=(65536, 65536), 3D=(4096, 4096, 4096)
Maximum Layered 1D Texture Size, (num) layers 1D=(16384), 2048 layers
Maximum Layered 2D Texture Size, (num) layers 2D=(16384, 16384), 2048 layers
Total amount of constant memory: 65536 bytes
Total amount of shared memory per block: 49152 bytes
Total number of registers available per block: 65536
Warp size: 32
Maximum number of threads per multiprocessor: 2048
Maximum number of threads per block: 1024
Max dimension size of a thread block (x,y,z): (1024, 1024, 64)
Max dimension size of a grid size (x,y,z): (2147483647, 65535, 65535)
Maximum memory pitch: 2147483647 bytes
Texture alignment: 512 bytes
Concurrent copy and kernel execution: Yes with 2 copy engine(s)
Run time limit on kernels: No
Integrated GPU sharing Host Memory: No
Support host page-locked memory mapping: Yes
Alignment requirement for Surfaces: Yes
Device has ECC support: Enabled
Device supports Unified Addressing (UVA): Yes
Device PCI Bus ID / PCI location ID: 66 / 0
Compute Mode:
< Default (multiple host threads can use ::cudaSetDevice() with device simultaneously) >
> Peer access from Tesla K40c (GPU0) -> Tesla K40c (GPU1) : No
> Peer access from Tesla K40c (GPU1) -> Tesla K40c (GPU0) : No

deviceQuery, CUDA Driver = CUDART, CUDA Driver Version = 7.0, CUDA Runtime Version = 6.5, NumDevs = 2, Device0 = Tesla K40c, Device1 = Tesla K40c
Result = PASS

这个机器有两个K40c GPU

5, amber14的GPU版编译

1，单GPU版编译
cd $AMBERHOME
make clean
./configure -cuda gnu
make install

测试
make test.cud

2，多GPU并行版编译

cd $AMBERHOME
make clean
./configure -cuda -mpi gnu
make install

测试：
export DO_PARALLEL=’mpirun -np 2′ # for bash/sh
setenv DO_PARALLEL ‘mpirun -np 2’ # for csh/tcsh
./test_amber_cuda_parallel.sh