Talking towards Ubuntu 10.04, server edition, what tools/practices would you recommend to secure the server?
This a bit non-specific, but in general you will need to
Run a firewall like iptables or ufw to manage connection to open ports.
Only install software your require.
Only run services that are essential to the running of the server.
Keep that software up to date with all security patches.
Set up new users with the least privileges they require to perform their duties.
Run denyhosts or fail2ban to check for brute force attacks.
Run logwatch to email you of any anomalies in log files.
Check your logs often for suspicious activities.
Use sudo always and use strong passwords.
Disable weak and medium strength ciphers in SSL for apache, exim, proftpd, dovecot etc.
Set services to only listen to localhost (where appropriate).
Run chkrootkit daily.
Run clamscan as often as is required to check for windows viruses (if appropriate).
Be vigilant, know your server, know what it should be doing and what it shoudn't be doing.
You will only keep things secure by constantly checking and securing. If you don't know what something does or how or why, or something looks suspicious, just ask others for advice.
Awesome answer by Richard Holloway. If you are looking for a specific step by step guide checkout the following 2 part guide from Slicehost library.
I use it almost everywhere when I have to setup an Ubuntu Server instance. I am sure you would love it.
Other great source is the Linode Library at http://library.linode.com/
Do check out the articles at both places. Loads of informations is available there and you will be armed with enough knowledge to handle your server just fine.
PS: In no way, a library can be a substitute for a great sys admin's intuition, insight and decision making capabilities.
Three things I tend to recommend are:
Mount all globally writable areas (/tmp, /var/tmp) as 'noexec': This for the most part is safe without quirks, except (as of writing) unless you choose to upgrade your system. See bug #572723 on Launchpad for more details there.
Don't install any compilers or assemblers unless absolutely necessary: I think this self explanatory.
Get started with AppArmor: AppArmor can be seen as an alternative to SELinux, and is great feature of Ubuntu to sandbox running applications to ensure they don't have any more access than what they need. I recommend reviewing the guide on the forums if you are interested. http://ubuntuforums.org/showthread.php?t=1008906
- Install and configure iptables with an appropriate ruleset for your environment. Filtering both inbound and outbound traffic.
- psad to detect and alert about any port scans against your system.
- Use fail2ban to prevent brute force login attempts against SSH.
- Disallow remote access using the root account, as this amongst other things means that if an attacker is going to attempt to brute force access to your server they have to workout both the username and the password.
- Use strong passwords for all user accounts.
- Limit SSH access to only be available from certain IP addresses if possible.
- Use Tripwire of another Host-based intrusion detection system.
- Monitor the server with a network monitoring program like nagios.
If I were you, I'd look into iptables (standard Linux firewall) and see what services are running. Basically you want to only be running the services you need, i.e. not running a web server when you just want to setup an email server, and to only have the ports open that you actually require. Everything else should be locked down!
Guide to iptables: https://help.ubuntu.com/community/IptablesHowTo
Hope this helps!
p.s If you need more help get on irc and hit the #ubuntu-server channel on freenode