In Ubuntu 20.04, to do what you want, I had to set both:
gsettings set org.gnome.shell.app-switcher current-workspace-only true
gsettings set org.gnome.shell.window-switcher current-workspace-only true
(the first is non-default in 20.04).
You can do this from the command line, or by installing sudo apt-get install dconf-editor
then running dconf-editor
(see detailed guide)
I found that the reload button in dconf-editor wasn't sufficient - I had to reboot (perhaps re-login would have been sufficient?).
With the defaults (window-switcher current-workspace only, but app-switcher not), Alt-` gives the contrary behaviour where the app-switcher is confined to the current workspace but the window-switcher within each app is not. So, for example, if you have browser windows open in other workspaces, you will have to cycle through them all. Setting the app-switcher to current-workspace-only should limit both to the current workspace.
According to the 19.04 release notes, Alt-Tab was changed from app-switcher to window-switcher. So, from 19.04 onwards (only tested in 20.04), I suggest you use Super-` and/or Super-Tab. You can hold down the Super while tapping either of the other keys in order to cycle through apps and/or windows (see switching between windows).
Super-` gives the same contrary behaviour unless you confine both the app-switcher and the window-switcher to current-workspace-only.
Note: Super is the Windows key and ` will often be whatever key is above Tab, whether or not it is labelled with a backtick or grave accent.