Just bought a brand new HP pavillion all in one desk top. I saved the current Ubuntu to a USB stick and Im not sure once im in setup what to do to have it boot up using the USB stick ?
Once you have made a bootable USB of Ubuntu installation medium, here's what you do.
You have to tell your BIOS to use the USB drive as the boot medium. This is probably not the default in a new system so you'll have to do it. Here's a link to an HP document on setting the boot order: https://support.hp.com/ca-en/document/c00364979.
When the computer is starting up it looks to this list of boot mediums to find which medium to try first. For example you could set the order to USB,DVD,Harddisk. The system will look down this list to find the first one with a bootable medium.
The USB option may not appear as an option in your BIOS screen unless you have a bootable drive in one of your USB slots. So before you start your computer, put your bootable USB drive into a slot. You have only a few seconds to press the BIOS entry key so be ready to press it right away.
Once you're in the BIOS dialog system your BIOS screens will tell you how to change the boot order. The link above will help.
NOTE that it is not enough to just copy the iso to the USB drive. You will have to use a utility that creates a bootable image. The Ubuntu site has clear instructions on how to do this. The comments to your question give many reliable sources.