To get the behavior you want you need to do a few things in addition to stopping the pop-up delay:
Install CCSM if not installed already.
apt-get install compizconfig-settings-manager from the terminal or search for "Advanced Desktop Effects Settings (CCSM)" in the Ubuntu Software Center.
Select the System Settings option in the power button menu (top right corner) and launch CCSM from there.
For Ubuntu 13.04 (Ringtail)
This may also work with > 12.10; unfortunately Unity changes significantly between versions.
Select the 'Ubuntu Unity Plugin' in 'Desktop'.
Under the 'Switcher' tab un-tick 'Show live previews of windows in the switcher'.
I'm not sure the popup delay exists in 13.04; I don't seem to notice any on my machine at least.
This section also allows you to swap the bindings of the switcher for all viewports and the viewport switcher.
For Ubuntu < 12.04
Select the Static Application Switcher in 'Window Management'.
Under the 'appearance' tab check 'icon' and 'show icon only'
And as @fossfreedom said, you can get rid of the .2 second delay "feature" under the 'behavior' tab by reducing 'popup window delay' to 0.
If you don't want them in rows you can switch to the 'application switcher' and change the settings similarly. Just make sure you do this by de-selecting the 'static application switcher' first, or else you'll get binding conflicts.