Though not a fan of apple, the observation in a SOLVED thread (on apple.com) that switching on the Caps Lock light (or keeping it lit? whatever... ) could be causing a delay might be a pointer into the right direction: the work-around suggested in the thread involves using 3rd-party software to emulate certain key combinations.
What comes to mind as an equivalent in the Linux world is xdotool
(see the man page, which has some basic examples). Specifically to your case, you may want to try sth like xdotool
key
Caps_Lock
as used in yet another SOLVED thread (on ubuntuforums.org).
According to my own experience (unless you're willing to really go a level deeper) you'll have to play around with different key combinations and xdotool
to find the magic combination which really does work smoothly. But definitely worth giving it a try!