The biggest difference between different Linux distributions is the package management system used. Ubuntu is very similar to Debian and uses the Debian package management system (.debs, apt-get etc.). However, there is a much bigger difference between Ubuntu and Fedora and Red Hat which use the Red Hat package management system (.rpms, yum etc.). ArchLinux uses the Pacman package manager. Other distributions, such as Gentoo, require everything to be built from source.
Distributions with a package management system often have a different set of packages available in their repositories.
The most visible differences are the default application selection and the default themes and settings. These seem to be the decisions that cause the most controversy but they are the easiest to change; all distributions are extremely customisable.
Also, the different distributions have limited binary compatibility, which means that the source code for packages has to be compiled separately for each distribution to create native binaries.
You can read about binary compatability her: https://wiki.ubuntu.com/MarkShuttleworth