When I would like to pause/play my radiotray application used the required shortcut the next window can been seen:
How can I solve this problem? Sometimes the shortcut works, but I do not know why.
Your media keys are likely working fine. However, they control your default music player which—judging by the overlay—isn't launched, or has nothing in the playlist.
RadioTray can't be added as the default music player, unfortunately, because it doesn't support the MPRIS API Ubuntu uses for extra-application media controls. The status of that issue can be tracked here, it seems.
@Regisz: Same here, I have Audacious as my default media player, but it doesn't work with multimedia buttons on my wireless keyboard or any keys on my headset. As soon as I start Rhytmbox while having Audacious still running, I can "control" both media players with a single Play/Pause button somehow.
Anyway, this question could be a duplicate of:
BTW, I've just found out that the new version of Audacious has own plugin for multimedia keys, so my similar issue has been resolved with it. Now it works!