As per the OP using
sudo apt-get remove --purge deluge is the correct way
If you have installed from source files or you need to find leftover Deluge files the Deluge wiki guide has details: RemovingFromSystem
The most likely reason for it still appearing in Dash is due to a leftover
You can use
locate deluge it will quickly show every file/folder that is named deluge and you should be able to determine if the application has actually been removed.
If you installed Deluge from source then you will manually need to remove the files. The main deluge application folder is usually installed in the python site-packages or dist-packages folder which should be seen in the output from the above command. You will also need to remove the deluge binaries from /usr/local/bin/ or /usr/bin/.
As neon_overload mentions, if you are having problems with Deluge, backing up and removing your old config may help fix upgrade issues. The default location is: