That is, I want to be able to pluck them out anytime without worrying if data will be lost. I want every write to actually be written immediately.
Mount them with the flush option. This is done by default in Ubuntu.
The 'flush' option is a middle-ground option. It will improve your chances over a default mount, however, there is still some caching. AFAIK, it is meant to provide a balance for flash drives.
If you want to go even farther, use the options 'sync' and 'dirsync'. These have drawbacks, though. You may take a huge performance hit since flash drives have to erase an unused block and then write that entire block with the old block's data and changes. This erase block size is hardware dependant and often much larger than the io block size. You could end up in a situation where the drive has to write 4MB just to replace one bit.
Not only can synchronous io tank performance, but the huge increase in block erase/write cycles will drastically reduce the life of your drive.
Nothing is going to protect you from data loss within a file that was open by an application and in an inconsistent state, ie., not closed, when you pull the drive, either.
Finally, the drive may have cache in hardware. Setting it to write-through may also be hardware dependent. I wouldn't be surprised if a sizable number of usb flash drives have cache and don't even report it to Linux.
One non-cache related option that increases write speed is 'discard'. This causes flash drives which support it to immediately erase or 'trim' blocks which are no longer used by a file or the filesystem once they are freed. Since you want to get data out of cache quickly, this will ensure you have plenty of erased blocks ready for writes, speeding io when you need it.