If you are running a web server on the same host, which means PHP is also probably installed, then you can install a tool that I wrote, use, and actively maintain:
Clients that need SSH access run the client portion as a system service on their OS of choice. The client keeps the server's SSH port and any other protected ports (e.g. POP3/IMAP) open with infrequent, encrypted packets. It is a pretty nice fire-and-forget solution. I've personally had zero issues with the software, but you might want to test it on a DigitalOcean droplet or similar service before deploying it to your production environment. Ports are kept open for up to 30 minutes (the default) unless renewal packets are sent (the client attempts a renewal every 10 minutes by default), which allows for IP addresses to change with minimal downtime. When ports are opened for an IP address, e-mail notifications are optionally sent to whoever is configured to receive such notifications.
The goal is to dramatically limit the attack surface by closing sensitive port(s) to everyone who does not have the encryption keys for clients on dynamic IPs where setting up a complex VPN is not an option. Even if an attacker winds up on the same unlocked IP address (e.g. a laptop on a coffee shop's WiFi), they still have to get through SSH, but at least Russia, China, and North Korea aren't able to even see the port.
By the way, I don't use ufw. It pollutes iptables with a bunch of unnecessary chains, makes iptables rules way more complicated than they need to be, and more rules equates to slower packet processing through iptables. I wrote a post not too long ago on writing elegant iptables rules:
It's actually a lot easier to use iptables directly and is less awkward than ufw when you can copy-pasta good, clean rulesets. Plus, you never know when you'll be stuck on a CentOS box.