I set Banshee up as the default app to launch when I plug in my Android device, but have changed my mind. I'd prefer that Ubuntu simply mount it as a drive and leave the rest to me.
How would I go about changing this behavior?
You could try changing the default behaviour for plugged in devices in Nautilus settings (the tab is called Devices I believe).
Like "Insert Audio CD" --> "open player" etc