One thing I really hate about Ubuntu is how easy it is to accidentally delete files.
Is there any way to show a pop-up confirmation dialog when the Delete key is pressed, before a file is actually moved to the trash?
Nautilus there is. Right click a file and move it to trash or deleting files.
The command line add a few aliases in ~/.bashrc
Add a few aliases and activate noclobber
No clobber prevents you from redirecting and overwriting to an existing file.
I do not know any way of over riding the delete key. You may be able to map the delete key to a script, but, as I would assume you would use the delete key for other tasks, you would then map a key combination to perhaps a zenity script.
Perhaps someone else will add a way to over ride the delete key.
The lack of an option for this is a result of the Gnome philosophy of striving for simplicity at the expense of easy customisation.
I would recommend switching to another file manager which has this option, like Dolphin or Thunar.
The default file manager is also difficult to configure in Gnome, but there are instructions to do so on the Ubuntu wiki.