What are the differences in feature set between the main Linux kernel and the kernel being used by Ubuntu? Why is Ubuntu branched off of the main kernel instead of using the main kernel directly?
The upstream kernels (referred, when we provide them, as mainline kernels) are simply the upstream kernel for the specified version, with a very few minor configuration changes added so that there is minimal issue using them on an Ubuntu Install.
The difference between these kernels and the one released for an Ubuntu version are numerous other configuration changes, source patches* and, in some cases, compiled in modules and hardware support.
Very little of the underlying code is different from upstream other than that noted above.
*: source patches are, in most cases, changes to the default behavior of the kernel. These are not limited to: blacklists, video quirks tables, built in modules, source changes that are not yet in Linus' tree or the stable queue quite yet etc.
Source patches are what the team uses to introduce configuration options that are required to support the vast array of released software for a specific version of Ubuntu. In many cases they are necessary for those applications to work properly. The fundamental difference between this and the upstream kernel is that the pure upstream kernel is not distribution specific.
Therefore, the configurations are not geared toward ensuring that certain applications work as expected. These configs exist to provide that level of changeability to the kernel for specific things. Additionally, the use of quirks allows us to identify hardware that requires specific things from the kernel that we can set once we detect them.
This is one of the major features of Linux and it is one of the reasons we have been able to run on so many diverse platforms over the years. There are times when the quirks and source patches cause issues between each other. During these times, we work to mitigate the problems with changes to underlying kernel behavior. In many cases these changes don't make it upstream until after the release due in very large part to the need to properly code the fix, document what was done, test the changes and conduct various other tests and stress testing.
In these cases, patches that are headed for upstream are carried as source patches in much the same way pre-stable and patches that are not yet in Linus' tree do. The link to the delta review spec should serve as a reference for what was reviewed for the current Natty release of the Ubuntu kernel.
The Linux kernel is in constant development. When packaging their next release, at some point in time the Ubuntu team has to say, 'OK, we'll take the kernel as it now stands' From time to time you may get a new kernel made available in the repos to use in the current distribution.
This place can make for some interesting reading but it can be highly technical. You may be able to work out what's cooking upstream and what you're currently missing out on (but will eventually get).
It should be noted, like any piece of software, a kernel is prone to be bug-ridden and it's often the case that regressions occur (new stuff added breaks old functionality). Kernels these days IMHO don't really excite me in terms of new functionality. I'd quite happily run a kernel that was two years old for example.
You can download the git tree of Ubuntu, and use a graphical viewer to view the git tree. That will show you all the patches of Ubuntu. https://wiki.ubuntu.com/Kernel/Dev/KernelGitGuide
Another possible option, subscribe to the ubuntu-kernel mailing list. All their work is posted there, and you can get an idea of what they do. But, it is high traffic, and IMHO, annoying. (But hey, it is their list)