It looks like you are experiencing this bug on bugzilla - and reported also on Launchpad.
Upstream says this is fixed - however this has not been backported on Natty.
Whilst I don't normally recommend installing deb packages from different versions of Ubuntu - in this case, the package is simple without dependencies - and the launchpad bug recommendation is similar.
I recommend that you download the oneiric deb (do not add the oneiric repository) and install it.
i.e. 32bit from here, and 64bit from here.
sudo dpkg -i <name of the deb file downloaded>
apt-get install xfce4-xkb-plugin
which another info do you need ?