Yes,It is possible. Here I explain the way I usually do, since I have same situation.
Limit is that you
must have same architecture of systems.
First Clean Cache using following command in terminal.
sudo apt-get clean
Dwonload packages for Install Ubuntu desktop using this command
sudo apt-get install -d ubuntu-desktop
This command will download packages only.
Now go to
Filesystem > var > Cache > apt> achieve
You will get all downloaded packages here. Copy all to folder.
Copy that folder to Usb drive.
Now Insert usb that server you want to install ubuntu desktop.
Type following commands.
cd /media/<your USB drive>/<folder>
sudo dpkg -i <all .deb packages>.
This will install .deb packages in System. You can install any .deb packages in this way.