I saw other answers but it didn't help much. Here is how I did it.
You need to download all three files from Nvidia Developer site. https://developer.nvidia.com/rdp/cudnn-download
but problem is that you need to
wget this files.
Well, you can do it from here:
Use Ctrl+F to search for exact file names which you can get from links from Nvidia Developer site.
That ftp site doesn't have DOC .deb file (last one - libcudnn7-doc_126.96.36.199-1+cuda9.1_amd64.deb). You can download it from here: http://apt.cs.uchicago.edu/pool/main/ c/cudnn/ libcudnn7-doc_188.8.131.52-1+cuda9.1_amd64.deb
Thank you University of Chicago :).
After that it's just matter of following installation guide from https://docs.nvidia.com/deeplearning/sdk/cudnn-install/index.html:
From directory where you just downloaded those 3 files containing cuDNN install the runtime libraries, for example:
sudo dpkg -i libcudnn7_184.108.40.206-1+cuda9.0_amd64.deb
Install the developer library, for example:
sudo dpkg -i libcudnn7-dev_220.127.116.11-1+cuda9.0_amd64.deb
Install the code samples and the cuDNN Library User Guide, for example:
sudo dpkg -i libcudnn7-doc_18.104.22.168-1+cuda9.0_amd64.deb
Use your own main, dev and doc files instead of those in examples.