+1 for the question which affects many of us. I can only say the following:
Even with today's technology and security measures there will always be a way to bypass them, be them an easy task or a difficult nearly impossible one. At the end, they can be done (See in google stuff like wikileaks, information leak, etc..). With this in mind I will add that having an SSL connection (Secure Socket Layer, although I am guessing a TLS 1.x will be used) is very secured. Even more in Ubuntu (Or any other Linux Distro) since, not only are the standards applied but they even get updated faster and with any new security measures.
Add to this that you will be using a VPN will give you even more security.
Having that, plus a secured OS like Ubuntu (Very secured since you are going to load it from a Live USB, which is like a Paranoid Security Level PSL) gives you a HUGE security measure over the PC you would be using. Of course the only downside is the connection to the place you would be connecting and the connection to it.
You can have a very secured system, but if the connection to the other end is compromised, it lowers your security. It lowers even more if where you are connecting has already been compromised.
So from what I can read from your question, the only 2 things left to think about would be:
- Am I connecting to a secure site?
- Is the connection I am using secured (Like somebody capturing packets like crazy)
- Do I have somebody behind me with a gun?
This are questions that can help you when having a more secured way of connecting. Now a good question to ask yourself is:
- Do I need so much security just to check my bank account that holds 100$?
- Am I Bruce Wayne that I need so much security?
I can give you an example of my father that uses Ubuntu + Firefox to connect to Bank of America and other 3 banks. He pays all bills, does transfers, payse credit card, etc.. This is like 3 years ago (Since he got Ubuntu ^^).