This is a known issue with the current implementation of Zeitgeist in Natty & Oneiric - the software that drives the Dash searching capability.
Zeitgeist Privacy has finally landed in 12.04 (Precise) - it has not been backported to Natty or Oneiric.
By default, privacy hides hidden files. That's the good news.
However, if you have got your files in a hidden folder then those files are not hidden.
Thus to hide a hidden folder and its contents you need to explicitly hide that folder.
To do this, launch Privacy from the System Settings
Click the button shown and this will show the directory window
Navigate to the folder below where the hidden folder is located.
Press CTRL+L to reveal the location bar and enter the name of your hidden folder.
Once you select the OK button, no further activity will be recorded in your hidden folder.
To remove current activity, use the linked question for further help.
- How can I keep recent files from appearing in Unity?