If you want to do it right, you should learn to use and love the terminal. Don't know how well you know it now, but the bash terminal is a great thing to know, and has lots of features (type man bash and see if you can figure it out, if you don't know it now. The learning curve is a bit steep, but once you learn, you can administer your server easily from anywhere).
There is good documentation on the Ubuntu site for setting up a server.
You should install the OpenSSH server (sshd), and the OpenSSH client on your desktop. That's all you need to log onto the server and do everything you need just like if it was local. You can run programs on the server, but display it on the client. You do not need Putty or any other software to do this.
Normally, you would install a database on the server, such as SQL server, which can be accesses by the Web Server, such as Apache. Users create accounts on the website, and they are stored in the database. You can install and administer all this using ssh.
As far as what to do after this, the Ubuntu guide will be good, but you don't need to install everything. Start with ssh, then the web server, database server, PHP, and possibly phpMyAdmin, but it depends on what you will be doing with the website.
The traditional LAMP server is Linux, Apache, MySQL, and PHP. This is what you usually get with a hosted website (non-windows one, at least).
You might want to go through a tutorial on creating websites, if you don't know what you are doing. There are some good hands-on tutorials, although I'd have to check on which ones are still around, since it's been a couple of years since I've done any websites. But I taught myself in a pretty short amount of time, and got jobs setting up and programming a bunch of sites for local companies.
EDIT: By the way, I personally would advise against installing LAMP as one installation. It's really not hard to set it up individually, and adding a level of indirection to me just complicates things. If something goes wrong, it's harder to figure out what it is. And when you need to modify something, it might be more complicated, or even not possible without learning how the stack works first.
