I've got pretty much the same problem. When scrolling the wheel very very slowly, it works okay, exactly five lines at the time in gedit. When rolling faster, it jumps forth forth back forth back at random with the effect that you often are at pretty much the same place after scrolling the wheel one turn. In Windows, it works perfect without a driver. It's a Thrust mobile BT mouse.
I've got a wild guess what's the problem.
First off, this is a wheel with discrete positions on the wheel. This means that the wheel will go fast faster fast faster fast faster when scrolling fast, because the discrete positions function as a break.
I suggest that the hardware inside the mouse is not good enough to distinguish between back and forth and fast faster fast faster. I know the inner workings of an electronic mouse and it support such an explanation.
When two lights shines through spokes in a photo optic receiver and the latter can only know more or less light and from this have to decide what direction the wheel goes, it can be hard enough to write software for this job when the wheel is turning at the same speed. The explanation why it's not a problem even in Windows generic driver is that MS has had the same problem with their own poor made mouses.
Solution
Write a mode in the driver in which it will turn into when detecting fast changes between back and forth at the mouse wheel. In this mode the driver will discriminate opposite direction from the last one, lets say, in a ring buffer with five last elements.
