From the man page for
sudo apt-get --purge remove package
to also delete the configuration files associated with that package.
You can also take a look in System->Preferences->Startup Applications.
On the other hand, you can always know what processes are currently running on your machine using System->Administration->System Monitor, clicking on the Processes tab. If you find a process which name is related to EC2, then you can
locate the binary (remember to
sudo updatedb before using it). To find out which package installed it, first enter
sudo apt-get install dlocate
Finally, remove the package using