I've seen in a few places that you can enable experimental features to get fractional scaling with the following command:
gsettings set org.gnome.mutter experimental-features "['scale-monitor-framebuffer']"
This doesn't work for me. When I run this command, there are no errors: I can see the correct value showing up in dconf-editor. But the fractional values don't show up in the settings. I can still only see 100%, 200%, and 300%.
I'm using Wayland, all Wayland apps. I'm using the 384.90 Nvidia drivers running on an Nvidia GTX 1080. Running stock, completely unchanged 17.10 (unchanged except for the above command).