I used to use File --> Connect to server to connect to my remote ssh a lot before 12.04/11.10. But now in a fresh install of 12.04, whenever I try to access the remote files, nautilus either always hangs or is ultra slow (>5 mins) in listing the directory contents. Most of the time I have to force quit or xkill. Mounting using SSHFS works fine. The ssh server is fast and it works fine via putty, mc (using fish) and normal terminal.
I even installed nautilus in Lubuntu 12.04 (virtualbox) and to my surprise, browsing ssh server was fast and smooth. Both versions of nautilus are at 3.4.2 thus I am led to believe the problem might be lying somewhere in Ubuntu 12.04. But I am clueless. All suggestions welcome. I really need to solve this problem.