alacritty is a terminal emulator. It's the drawing on your GUI that lets you do thing as if you were on the console.
tmux is a terminal multiplexer like
screen that lets you run multiple terminal sessions within a single terminal but that can remain active even if you end your terminal application or connection.
You can connect to
tmux from within
alacritty or any other terminal emulator or ssh, etc connection. You can even connect to the same session from multiple terminals.
alacritty however you start it.
tmux attach || tmux new to attach to an existing session or start a new one.
tmux attach -t target-name || tmux new -s target-name to attach or create to a specific session.
Alternatively a script file:
tmux attach -t MyFavSession || tmux new -s MyFavSession
Create a shortcut or edit the Alacritty menu entry to...
alacritty -e $HOME/bin/tmuxed
...to start alacritty with the tmux attach/new command automatically.