There isn't one command that you can run which will easily clean up all the already-deleted files for you. However, there are a number of things you can do to reduce your vulnerability to this sort of attack in future.
As others have said, using tools like shred or srm allows you to delete a specific file by actually overwriting it, rather than just removing it from the filesystem. If you're feeling bold, you can replace the
rm command with
srm to securely delete files going forward. That means that whenever you (or another program) tries to delete something using
rm, the secure delete command will run instead.
However, if you're using a solid state disk, or even some newer mechanical disks, shred and other overwriting-based methods may not be effective, since the disk may not actually write where you think it's writing (source).
A more convenient option is full-disk encryption. If you use the alternate installer, Ubuntu can automatically set up a fully-encrypted disk for you you, but you can also customize and configure the settings yourself. Once installed, the encryption is almost invisible to you: after you enter the passphrase (be sure to pick a good, long one) when the computer starts up, everything looks and feels just like normal Ubuntu.
You can also encrypt external media like USB drives using Ubuntu's Disk Utility. Setting up an encrypted external disk is as simple as checking the "encrypt underlying filesystem" box when formatting the disk. You can even store the passphrase on your (encrypted) keyring, so that you don't need to enter the phrase every time you plug that disk into your computer.
If your whole disk -- and all your removable media -- is encrypted, there's much less to worry about. A thief or police officer would need to swipe your computer while it's on, (or within a minute or two of turning it off if they're very good) in order to access your data. If you hibernate (rather than suspend) your computer when it's not in use, then you should be pretty safe.
If you ever need to completely destroy all your data, you don't need to do a Gutmann wipe of your whole disk. Simply overwrite the very beginning of the disk, to destroy the headers for the encrypted volume. Unlike with a regular filesystem, this will actually make it impossible to recover the data.
So, how do you go from your current setup to a safely encrypted disk? It's quite a challenge to retrofit a currently-installed operating system to use an encrypted disk. The easiest approach is to backup all your data and settings, then reinstall with an encrypted disk. When backing up, make sure to back up your data to an encrypted external drive, but don't save the passphrase in your keyring.
After you've backed everything up, you may want to aggressively wipe your hard drive, to make sure that none of your existing data can be recovered in the future. If you're using an SSD, the process is even more challenging, so depending how much you want to invest in the process, it might be worth destroying your current disk (a challenging proposition) and starting with a new one.
When reinstalling the OS, if you haven't aggressively wiped the disk already, you should make sure to completely fill the new encrypted partition, which will overwrite all your old data. Once you've restored your backup, you may want to aggressively wipe the start of the backup disk, to destroy the encryption header, so that it can't be recovered again.