It is not as easy as it looks at first sight to acchieve this. See the discussion below for reasons why it is in fact rather complicated:
The existing modules
module-switch-on-port-available will not work consistently with HDMI.
At present there is much effort being done to implement a priority list feature which will let us define the priority of an audio device for pulseaudio to route to it (or not):
Until this will be generally available in a future release of PulseAudio we have not much more choice than manually switching the audio output sink whenever we connect to a HDMI audio device.
To make life easier in case we often have to switch our output devices we may consider writing a small script to enable toggling between two output sinks. See these questions for a start: