After sending a message to the wacom developers mailing list:
The two settings used above can not resolve the issue. It is a
bug in X server/XInput.
The problem lies in the difference between x, y scales. We used to
unify the scale in wacom_drv.so when mapping and screen information
were considered inside the driver (in linuxwacom package). The
evidence of this work is still in the comments of xf86-input-wacom.
Refer to line 611 of wcmCommon.c[*]. However, the actual scale match
can not be fully implemented inside the driver any more since the new
factorX/Y does not have screen info.
Screen information and relative mode acceleration are pushed out into
X server. Mapping and scale unification should be processed by in X
So, by design, the scale is unified so users can draw circles in
relative mode, without changing settings. - Ping Cheng (Full Email)
And then from Peter:
yeah, that's a bug in the server, this should be exposed by the ptraccel
code but it isn't yet. I've never found the time to properly investigate
where it's going wrong, but you can even see this behaviour when you add a
monitor - suddenly the acceleration factor goes up by a fair bit.
As Ping said, this cannot be fixed in the driver, we need the server to
handle this correctly. - Peter Hutterer (Full Email)
In conclusion, I think the best action is to report the bug and champion a developer to sort through the code to get a fix in place.