First check the status, it should look something like this:
$ sudo pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.1 main 5432 online postgres /var/lib/postgresql/9.1/main /var/log/postgresql/postgresql-9.1-main.log
9.3 main 5433 online postgres /var/lib/postgresql/9.3/main /var/log/postgresql/postgresql-9.3-main.log
Stop the old version:
sudo systemctl stop [email protected]
Then make sure to test that you can reach the new version, and that your data is actually there (even if pg_upgrade reported no errors, please check!).
To delete the old version, including the old data directory, run:
sudo apt purge postgresql-9.1
You should now only have the new version left:
$ sudo pg_lsclusters
Ver Cluster Port Status Owner Data directory Log file
9.3 main 5433 online postgres /var/lib/postgresql/9.3/main /var/log/postgresql/postgresql-9.3-main.log