The detailed answers are very good, but (aside from software that may have hard coded absolute paths in it - not the best programming practice), the main point is that non-system/non-distribution software should not be stored mixed in with the regular system files.
Putting things in
/usr/local keeps things clean and safer.
In particular, your software search path ($PATH) determines the order in which locations are searched when looking for a program of a particular name to execute. Usually, places like
/usr/local are toward the end of the list.
If you install a package that has a program named
cp in it, the default search order that comes with your distro will find the normal one because the directory it is stored in is searched before places like
If it didn't work that way, who knows what might break or open a security hole if a program named
cp that does something else gets run when you think you're just trying to copy some files.
If something like this does happen, it may take awhile before somebody thinks to run a command like
type cp (which might not even be enough to show that something is wrong) to find out that what is being run isn't what you think it is. Until that point, you're stuck at "Everything is exactly the way it should be aside from the small detail that it doesn't work!"
It basically helps keep unexpected things from happening and also avoids situations where system updates might remove or replace some or all of your "custom" installed packages. Or, the reverse, some "custom" programs might overwrite system supplied programs that many other programs or scripts may rely on.
From an administrative point of view, mixing "system" and "optional" programs/files in the same locations places your system in an "undefined" or at least "ambiguous" state.
If you have a problem with your system or a program and need help, one of the first questions that gets asked is "What did you change?" and "Can we temporarily disable some of all of those changes so we know we're looking at the real problem and not just a symptom of something else."
With separate locations, these changes can be quickly identified and all you have to do (at least for the programs themselves) is temporarily remove their directories from the path.