I have the latest CUDA toolkit and drivers installed on a 12.04 server. The drivers are working fine: all the NVIDIA sample code compiles and runs and I've written, compiled, and run several CUDA programs. The other day I went to use the new nvprof command line profiler and was greeted with the following error:
Error: unable to locate profiling library libcuinj64.so.5.0.35.
I double checked the CUDA libraries and that specific library is in fact included in the LD_LIBRARY_PATH. (I used the method described here). Running
ldconfig -v shows that all the Cuda libraries are loaded into LD_LIBRARY_PATH:
/usr/local/cuda-5.0/lib: libcurand.so.5.0 -> libcurand.so.5.0.35 libnpp.so.5.0 -> libnpp.so.5.0.35 libcusparse.so.5.0 -> libcusparse.so.5.0.35 libcufft.so.5.0 -> libcufft.so.5.0.35 **libcuinj32.so.5.0 -> libcuinj32.so.5.0.35** libcudart.so.5.0 -> libcudart.so.5.0.35 libnvToolsExt.so.5.0 -> libnvToolsExt.so.5.0.35 libcublas.so.5.0 -> libcublas.so.5.0.35 /usr/local/cuda-5.0/lib64: libcurand.so.5.0 -> libcurand.so.5.0.35 libcuinj64.so.5.0 -> libcuinj64.so.5.0.35 libnpp.so.5.0 -> libnpp.so.5.0.35 libcusparse.so.5.0 -> libcusparse.so.5.0.35 libcufft.so.5.0 -> libcufft.so.5.0.35 libcudart.so.5.0 -> libcudart.so.5.0.35 libnvToolsExt.so.5.0 -> libnvToolsExt.so.5.0.35 libcublas.so.5.0 -> libcublas.so.5.0.35
The offending library is in bold. At this point the old command line compiler works fine, the compiler itself works fine, so it seems to be nvprof specific.
I'm not sure if this is Ubuntu or the nvprof build provided by NVIDIA. Does anyone have an experience with running nvprof under 12.04 or even 12.10? Anyone seen this problem before? I realize this is a fairly niche question but you never know.
UPDATE: This problem seems to persist on Ubuntu 12.04 with CUDA 5.5 (released on August 1, 2013). The fix still works, you just have to use the updated library (see below).