USB in different versions of Virtual Box
For use of USB in Virtual Box 3.x you need a PUEL-version. From Virtual Box 4.x USB 1.0 is supported in the OSE version installed from software center. For USB 2.0 we need to install an extension pack free for download from Oracle. This will make our Virtual Box a PUEL-Version (see this question on details on how to install Virtual Box from the Oracle repository).
Become a "vboxuser"
To be able to get access to an attached USB device we need to be in the vboxusers group. This can be done from Users and Groups after having installed the gnome-system-tools
or from the command line by
sudo usermod -aG vboxusers <username>
We need a reboot for group membership to take effect.
Activate USB support in Virtual Box Manager
We need to activate the virtual USB driver in our guest OS. In the USB settings from Virtual Box Manager tick "Enable USB Controller" For enabling the USB 2.0 driver also tick "Enable USB 2.0 (EHCI) Controller".

Select host USB device for access from the guest
To grant access to USB devices we need to select a device to disable in the host and to enable in the guest (this is a precaution to avoid simultaneous access from host and guest). This can be done from the panel Devices menu or by right mouse click in the bottom panel of the Virtual Box Manager on the USB icon:

Tick the device you need in the guest, untick it if you need it in the host. The selected device will immediately be accessible from the guest. A Windows guest may need additional drivers:

Use USB filters for permanent access in the guest
By defining USB filters we can define USB devices that will automatically be presented to the guest when booting the guest OS.

Click on the green + symbol on the right to add a known device.
Note, that some devices may lead to a boot failure of the guest. We can not use these devices for filters.
Read more on USB support in the Virtual Box User Manual.