It works for me to position the mouse pointer just a little outside the border of the window (above the scrollbar), then move it a little up or down (in a straight line, until it is outside the scrollbar), then the resize icon appears.
Alternatively, you can wait a few seconds while keeping the mouse above the overlay scrollbar, the overlay scrollbar will disappear (fade away), the resize icon will appear.
Edit
Toggling between scroll modes with a shortcut key combination
Since you mentioned in a comment that toggling between scroll modes would be helpful, you can simply toggle the scrollbar mode to "normal" and back to "overlay" by putting the script below under a convenient key combination. It recognizes the current scrollbar mode, and switches between "normal" and "overlay-auto".
How to use
Copy the script below into an empty file, save it as toggle_scrollbar.py
Test the script by running it from a terminal window with the command:
python3 /path/to/toggle_scrollbar.py
If all works fine, make it available under a shortcut key combination: System Settings > "Keyboard" > "Shortcuts" > "Custom Shortcuts". Click the "+" and add the command.
The script:
#!/usr/bin/env python3
import subprocess
get = lambda cmd: subprocess.check_output(["/bin/bash", "-c", cmd]).decode("utf-8").strip()
def set_value(cmd):
subprocess.Popen(["/bin/bash", "-c", cmd])
if get("gsettings get com.canonical.desktop.interface scrollbar-mode") != "'normal'":
cmd = "gsettings set com.canonical.desktop.interface scrollbar-mode 'normal'"
else:
cmd = "gsettings set com.canonical.desktop.interface scrollbar-mode 'overlay-auto'"
set_value(cmd)