I am attempting to create a 4GB partition on my Ubuntu boot partition. Under Gparted it says I have 26.19gb of unused space, how do I create a 4gb partition from this? The resize option is greyed out, and I'm worried if I unmount it that I won't have an OS.
You will have to resize the partition in order to create a new one, but because your system uses that partition, you can't resize it while you are booted into your system.
Make a Live USB/DVD (Like the one that you used to install Ubuntu), and boot onto that. Once you are on the live system, you can make changes to your partitions.
There is no safe way to resize your partition while you are booted into the system.