The middle mouse button is mapped to paste the current X-selection, which is normally whatever text is selected. This happens because back in the early days of gui's there was disagreement about how copy/paste should work. Some wanted there to be an explicit command to move something into the copy buffer, others wanted whatever was selected to be moved in there automatically. The makers of X11 (way back when these disagreements were still going strong in the 1980's) decided to implement both and make both sides happy, putting mouse-3 to paste the current selection and ctrl-c/ctrl-v for the copy and paste more commonly found today.
As to how to stop it the unfortunate truth is that you cannot without either patching X or disabling the middle-mouse button all together (which can be done by running
xmodmap -e "pointer = 1 25 3 4 5 6 7 8 9"
or by putting the line
pointer = 1 25 3 4 5 6 7 8 9 in your
My advice would be stick it out. The X-selection copy functionality is actually quite useful in many places and it is one of those things you quickly get used it.