The bare bones minimal would be
sudo apt-get install xfce4
If you do not install a log in manager (slim, gdm, light-gdm, etc) you will not have a graphical login screen, and in that case you can either startx
or xfce4-session
Depending on how much or little you wish to configure, you may want to go with lightdm ;)
If you need assistance configuring your xfce4 session (from the command line or with a log in manager) post back.
You will almost certainly wish to install additional fonts and additional xfce packages. Many of the extras are "plugins" , ie "xfce4-foo-plugin"
Either search for the packages, or if you prefer, install the "goodies" package
http://packages.ubuntu.com/precise/xfce4-goodies
Additional packages to consider (IMO) might be firefox, synaptic, squeeze, xfburn
Text editors , personally I prefer gedit as it has a number of features, but leafpad or mousepad are more traditional / lightweight (but scant features).
The following would give you a nice (IMO), minimal setup, adapt the packages as you wish.
sudo apt-get install xfce4 xfce4-goodies xfce4-mixer gedit squeeze synaptic xfburn lightdm audacious audacious-plugins
Other plugins listed here
http://packages.ubuntu.com/cgi-bin/search_packages.pl?keywords=xfce&searchon=name&subword=1&version=all&release=all
Add in a few fonts and multimedia as required and you would be good to go.