Here's some information from
man dpkg, which explains what
dpkg -r and
dpkg -P do. Removing a package with APT commands or synaptic involves the same procedures; as far as I know high-level package management utilities like APT and synaptic simply call DPKG as a subprocess to remove or install files. Their function is to calculate and resolve dependencies and provide an improved UI.
-r, --remove package...|-a|--pending
Remove an installed package. This removes everything except conffiles,
which may avoid having to reconfigure the package if it is reinstalled
later (conffiles are configuration files that are listed in the
DEBIAN/conffiles control file)[...]
Removing of a package consists of the following steps:
1. Run prerm script
2. Remove the installed files
3. Run postrm script
-P, --purge package...|-a|--pending
Purge an installed or already removed package. This removes everything,
including conffiles. [...]
Note: some configuration files might be unknown to dpkg because
they are created and handled separately through the configuration
scripts. In that case, dpkg won't remove them by itself, but the
package's postrm script (which is called by dpkg), has to take care
of their removal during purge. Of course, this only applies to files
in system directories, not configuration files written to individual
users' home directories.
Purging of a package consists of the following steps:
1. Remove the package, if not already removed.
2. Run postrm script.
As you can see, these commands delete all of the application's files with the exception of configuration files. Even
-P / --purge may not always remove local configuration files from users' home directories in some cases, so these files may need to be deleted by hand if they are no longer needed. When you mention application directories remaining after uninstallation, you are most likely talking about a local config directory.*
In my answer to a somewhat related question I discussed how APT, DPKG and other package managers installed on an Ubuntu system store information about the files provided by a package in a human-readable form.
That answer also tries to explain why using a package manager to handle package management is substantially easier than trying to do so by hand; in a typical Linux filesystem, files provided by applications are scattered all over the place. While this might seem confusing, there's a great security advantage to application files being in system directories owned by root that users cannot write to. Besides, if we do need to find out where files related to a package are, we can use various command line utilities to find them.
You wanted to know whether deleting all of the files of an application to uninstall it is a bad idea and if so why. My answer to this question is YES, in general it's a very bad idea, because deleting the installed files is only part of the work done by package management when you call it to perform an uninstallation. The DPKG database stores information on which packages are installed and installable, and all the relationships of dependency between them. If package files are removed without the database being updated, then dependency resolution won't be done, software that depends on silently missing software will fail to run, packages will be broken, the cleanup done by prerm and postrm scripts
dpkg calls for each package it removes will not be done, potentially leaving broken symlinks and incorrect config files lying around, and everything will be horrible.
Package managers exist for good reasons. Use them where possible rather than trying to do-it-yourself.
*However, sometimes software may be installed (and have a single directory) without DPKG knowing anything about it. For example, you may download a 3rd party application, extract the archive and run the executable file. To "uninstall" this software, you can simply delete the extracted files. Another example is installation using
sudo make install after compilation from source, which leaves the source directory behind. Uninstalling such software can be a headache if the developers have not configured things so as to enable
sudo make uninstall or not included a file manifest! A way around this is to create a deb package after
make, which can be done with