I already tried to right click on workspace switcher, but it doesn't show anything.
You can do that with Compiz. To install, just press Ctrl+Alt+T on your keyboard to open Terminal. When it opens, run the command(s) below:
sudo apt-get install compiz compizconfig-settings-manager compiz-plugins
Once installed, run the program, and navigate to Window Management --> Workspace Naming, and change the names to whatever you want.
I just installed using the command above on 14.04, and 16.04, and it works.