I'm writing this from an HP Stream Mini desktop running Ubuntu 14.04 LTS (64-bit), so I'm proof it can be done. (From what I've read the Stream Mini desktop is basically the same system as the notebook, just in a different form factor.)
The process isn't hard, but it's hard to find out exactly HOW to do it.
I'll assume you already know how or can find out how to put Ubuntu onto a bootable USB stick. The trick is getting the HP to boot to that stick.
Long story short: you need to press F10 at boot to get to the UEFI/BIOS configuration. Just change the boot order so your USB stick comes before the hard drive.
If you are unable to find your USB Drive, try switching boot options to "Legacy", saving those settings and retrying F9
A lot of stuff out there mentions pressing F12, but that just brings up UEFI boot options, which won't show the USB stick.
I am planning to do a full write up of this on my blog soon (I'll post a link here when I do), but for anyone with some experience installing Ubuntu, I'm guessing the only problem you'll have is getting the USB stick to boot.
It's also possible to tell the system to boot to the USB stick by navigating a labyrinth of options within Windows 8.1, but that only works until you've overwritten Windows. I've installed Ubuntu 3 times on this machine now. (Don't ask... it's not because it didn't work, it's because my experimentation got out of hand and I needed a clean start.)
I will say that once you get Ubuntu installed, it is smooth as butter. This is not a powerful machine but it handles Ubuntu really well. I have not run into any hardware compatibility issues.