From Virtualbox manual:
Most mainstream, modern operating systems, including Windows and
Linux, ship with support for one or more paravirtualization
interfaces. Hence, there is typically no need to install additional
software in the guest (including VirtualBox Guest Additions) to make
use of this feature.
VirtualBox provides the following interfaces:
Minimal: Announces the presence of a virtualized environment. Additionally, reports the TSC and APIC frequency to the guest
operating system. This provider is mandatory for running any Mac OS X
guests.
KVM: Presents a Linux KVM hypervisor interface which is recognized by Linux kernels starting with version 2.6.25. VirtualBox's
implementation currently supports paravirtualized clocks and SMP
spinlocks. This provider is recommended for Linux guests.
Hyper-V: Presents a Microsoft Hyper-V hypervisor interface which is recognized by Windows 7 and newer operating systems.
VirtualBox's implementation currently supports paravirtualized clocks,
APIC frequency reporting, guest crash reporting and relaxed timer
checks. This provider is recommended for Windows guests.
Ubuntu kernels are compiled with KVM guest support, as you can see in the output of grep CONFIG_KVM_GUEST /boot/config-*, so there's no need to change anything in the guest system.
I think you should install Guest Additions. I didn't run any benchmark but I felt the guest system more fluid and responsive with Guest Additions & KVM when compared to KVM alone. Also, without Guest Additions you won't be able to access shared folders or copy/paste between host and guest.