The most critical difference is security, transparency, and more granular control. This is why *.exe files are a very dangerous proposition to accept. In some detail:
In the windows world you have to trust that *.exe file. How can one really be sure it is to be trusted? How can you even know that it installs something? How can you be sure it doesn't do other things behind your back?
In Ubuntu, packages are digitally signed, so by using the package manager (be it muon, synaptic, aptitude, or even straight apt) - you get the content verified before it is even unpacked. This assumes, of course, that you trust the repositories. I'd rather trust the Ubuntu repositories (a single authority) than hundreds of often unfamiliar different sources to download from.
With a *.exe file you can do essentially one thing: execute it. In Ubuntu you can inspect the contents of the package, descriptions, configs, individual files, latest changes, bug fixes, etc. from the convenience of your package manager before deciding whether to install or not.
When you install from an *.exe file you also have to trust its 'uninstall' hook (and not all *.exe files are guaranteed to have one). In Ubuntu, all files belonging to standard packages installed by the package manager can always be uninstalled. The package manager is a separate and trusted application which provides both the installer and uninstaller, the package itself cannot take away the uninstall hook from you. Of course, a malicious package can sneak stuff via post-install actions, but that's why we have the official repository system and the same people we trust to maintain them.
It goes further. On Ubuntu I can really trust my system because I can verify the software on many levels. The ultimate level is being able to look at source-code. Packages have corresponding src packages. I can actually look at the source (Example: 'apt-get source bash' will give you the full sources to the bash shell). In the world of *.exe files, usually there are only binaries, and who knows what they actually do behind the scenes?
That said, there are always exceptions to the rules, but for me security and trust mean I cannot accept executing binaries from hundreds of different sources that are hard to verify as a standard way to install components on my system.