Part 1. Set swappiness. This may be as per degusa's answer, but it could be the opposite, and it'll have more effect when it is.
One scenario that some of us are happy to be in is when we have plenty of RAM. Generally, we've a small percentage of it being directly used by the kernel and applications, some (maybe a large amount if you've used other tweaks to boost performance such as mounting /tmp in memory) used for ramfs and tmpfs, and gigs and gigs being used as a disk cache to make our file-access faster.
In this scenario once the total used memory including cache becomes high, and an application needs more RAM, linux has to decide whether to take some cache from the file system, or swap out to the swap partition/file.
Since we've tonnes of RAM, and quite possibly only bothered with swap at all so we could enable hibernation, we want it to lean toward taking some of that copious cache, and hence want a low swappiness. If we don't care about being able to hibernate, we might even find that such a high-RAM machine doesn't need swap at all.
Another scenario is someone with low RAM who is switching between a few heavy applications and spending a reasonable amount of time on each. Imagine perhaps a web dev who spends some time on their IDE, some on a graphics editor, some on their browser of choice, a bit on some other browsers to check on compatibility issues, and maybe 5 minutes every hour on their mail client. They're also likely hitting the same files repeatedly with reads and writes and hence benefiting appreciably from file caching. This person could probably benefit from linux being more eager to swap out the memory used by whichever of those heavy applications they're currently not active on, so swappiness should probably be higher for them.
Not only is the best setting for them likely to be higher than the most common advice, but they're probably going to notice it more than the person who always has gigs to spare anyway, too.
Part 2. Priority & number of partitions.
Each swap partition has a priority, and linux will use that with the highest first. If not set in /etc/fstab, it'll be treated as negative starting with -1 (explicit settings are between 0 and 32767 and so -1 is lower than any explicitly set) and continuing in order in fstab to -2, -3 and so on.
The best setting depends upon where the partitions physically are. If you've only one, then it doesn't matter (but maybe you should have more than one, so read on).
If you've two or more on the same physical drive, then they should have different priorities so that it doesn't try to use two partitions that require seeking between them (does anyone know if this is automatically avoided?). The defaults are fine. It's probably not a good idea to have two swaps on the same drive anyway, but it can happen if you created one and then decided you needed more swap later (perhaps when adding more RAM).
If you've two or more on two or more physical drives that are of about equal speed, then setting them to the same priority will mean linux will use them both at the same time, which offers better performance for reasons analogous to why RAID or simply ensuring that there are frequently used files on both drives will - the work gets split between them.
If you've two or more physical drives of equal speed but have swap only on one, maybe you should change that, for the above reasons.
If you've two or more physical drives, of very different speeds, then generally you want the fastest drive to have a higher priority than the slower, so it's used first. You may not even want to have any swap on the slower, though it might make sense if you e.g. have a small swap on a fast but small drive for fast swap, and a larger swap on the slower drive so you've enough space to hibernate.
If the faster of the two is an SSD, then there's two alternatives with different pros and cons:
- Highest-priority, or perhaps only, swap on the SSD, for speed.
- Only swap on the non-SSD, to reduce writes on the SSD and hence increase its lifetime.
Number 2 is probably the one to go for if you only really have swap to allow for hibernation and otherwise your copious RAM means you don't really need it (and if you're spending money on an SSD, you should spend it on RAM too) unless perhaps you're a boot-up speed fanatic who wants to boot from hibernation at a speed that'll show off your fancy high-spec rig! Otherwise, the balance is all about speed vs SSD lifetime.
If you've a drive existing solely for swap so as not to compete with other file I/O, then you presumably are hard-core about performance and already know about this stuff better than me and are only reading this to see if I got anything wrong!