I first freshly installed Ubuntu 20.04 server and then xubuntu-desktop on top of it. I immediately noticed that many applications were missing main menus. The examples I remember were got-cola and libre office from the main repository. Also Microsoft Visual Studio Code (from Microsoft repo) didn't have it. Thunar at some point didn't have it but after me tinkering around got them back. There were a few other apps without menus but I don't remember what they were.
I already know the answer but I am posting it as a question if someone else is experiencing the same issues. I am giving my solution below and hoping it may help others.