On the remote server define a secondary IP address, for example 10.0.0.1, that points to itself. (The same way 127.0.0.1 does.)
On your local machine, create your sshuttle connection as follows:
sshuttle -r user@remote 10.0.0.1
# "user" is your username on the remote machine
# "remote" is the name or IP address of the remote machine
You can now access the remote server at the 10.0.0.1 address. Because it understands connections from 10.0.0.1 to be "itself", all the ports are accessible at that address. (They stop being accessible as soon as you close the sshuttle connection.)
You can confirm this with the following commands:
nmap remote # returns only ssh port
nmap 10.0.0.1 # returns all ports
You can now connect to your database with the address
10.0.0.1:3306. For example, if it's a MySQL database you can connect to it uing MySQL Workbench.
If this approach suits you, you may want to investigate using sshoot to manage your sshuttle connections.