My Ubuntu was installed with only a German keyboard layout configured. I much prefer the English UK (extended WinKeys) layout and therefore added that option.
However, the selected keyboard layout kept reverting to German seemingly at random, and so I removed it through the layout indicator's preferences dialog. Strangely that didn't help: every now and then the English keyboard layout is replaced by the German one and that change shows up in the layout indicator.
I guess the unwanted layout must still be configured somewhere and I must be hitting some key combination to switch to it. But the preferences dialog for the layout indicator doesn't show any such combination.
I'm on Ubuntu 12.04 and I'm using Xfce 4.8.
PS: This is similar but not the same as this or this issue: when I log in, the correct layout is usually selected (and only that layout is configured). The layout is changed completely at random while I'm working.