I would suggest you to remove the following packages :
- apache2 and related modules
sudo apt-get purge <package_name> to remove the packages.
Below text is grabbed from here:
Starting over: How to remove the LAMP stack To remove the LAMP stack
remove the following packages:
Note: This assumes you have no other programs that require any of
these packages. You might wish to simulate this removal first, and
only remove the packages that don't cause removal of something
apache2 apache2-mpm-prefork apache2-utils apache2.2-common libapache2-mod-php5 libapr1 libaprutil1 libdbd-mysql-perl libdbi-perl
libmysqlclient15off libnet-daemon-perl libplrpc-perl libpq5
mysql-client-5.0 mysql-common mysql-server mysql-server-5.0
To also remove the debconf data, use the purge option when removing.
To get rid of any configurations you may have made to apache, manually
remove the /etc/apache2 directory once the packages have been removed.