I am wondering: Is there more difference between installing Xubuntu from ISO and installing XFCE to Ubuntu? - If so which are these difference? - Which of the two do you recommend and why?
There is an xubuntu live (desktop) cd available here :
So when you install xfce you get xfce.
xubuntu, however, is an additional set of packages and customizations to make the xfce experience more rich. xubuntu-desktop is a meta package and you can see the entire list of packages here
As you can see the list includes xfce + a number of additional packages.
Use which ever you prefer, from your post I would suggest you stay with Ubuntu (unity).
Installing XFCE will give you the upstream XFCE desktop environment. The same way that Ubuntu differs from upstream Gnome, Lubuntu differs from upstream LXDE. For instance, Ubuntu uses Firefox instead of Epiphany and Thunderbird instead of Evolution. Installing XFCE or Gnome in Ubuntu will pull in extra applications that are normally used in those environments but isn't installed by default in Ubuntu. Other than that, the software is the same so there's no radical difference.
Xubuntu has Xfce with additional packages and default configurations that differ from Xfce's. You can see the difference by choosing the Xfce session instead of the Xubuntu session, at the login manager screen.
Most likely, you want Xubuntu/xubuntu-desktop; it would be an edge case if you specifically needed Xfce and not xubuntu-desktop.