In Ubuntu 12.04 LTS,
Goto the Keyboard Layout Options window I could select Key(s) to change layout and mark the binding of my choice. I use Shift + CapsLock to toggle the layout from the keyboard itself without having to use mouse cursor to select the required layout from the panel indicator. The same key combo seemed to work to serf through or select between multiple layouts. (But in Ubuntu 13.10 it wasn't so easy.)
In Ubuntu 13.10,
Goto the Keyboard settings window, under Shortcuts tab, I select Typing. The default for Switch to next source is Super+SPACE which didn't seem be working for me and so by clicking on it and I assigned a New Accelerator..., to change to next Layout (I used Super+L).
(Then I assigned something similar for Switch to previous source using the same preferred layout, and then began to test them. Switching from the preferred layout to the alternate worked but the reverse did not. Please refer the next step for the fix.)
The Fix: After assigning Super+L for "Switch to next source" using my preferred layout (A), I changed my keyboard layout to the alternative layout (B) and then assigned the combination to "Switch to previous source" (for which I used Shift+Super+L). Then I tested them, and they work fine. (Maybe disabling "Switch to previous source" would work too.)
The new accelerators for the key combos that I used looks like this:
I could then switch to layout (A) using Super+L and back to layout (B) using Shift+Super+L.