I wrote a script that can be used to clean install a desired version of cuDNN or upgrade an existing one. Once you perform the steps on the GitHub page, you can change your cuDNN version to v6.0. After that, you don't have to make any configuration for TensorFlow, it will start working automatically with cuDNN v6.0. You can find the script here.
This is what the script does:
rm -f /usr/include/cudnn.h
rm -f /usr/lib/x86_64-linux-gnu/*libcudnn*
rm -f /usr/local/cuda-*/lib64/*libcudnn*
cp -P packages/cudnn/include/cudnn.h /usr/include
cp -P packages/cudnn/lib64/libcudnn* /usr/lib/x86_64-linux-gnu/
chmod a+r /usr/lib/x86_64-linux-gnu/libcudnn*
rm -rf packages/cudnn
It performs installing operations automatically for the version you want.