Is it safe to delete the PPA after successful installation of the package?
Depending on what you consider "safe". Yes, you can delete any software repository from your sources.list you wouldn't notice. But, you also will be unable to upgrade the packages from that repository, which may offer bugfixes or security patches.
It is recommended that unless you have a very good reason, you keep the repositories on your sources.list (apt 1.1.10+ changes the strategy to update package lists, which improves performance) and allow upgrades. If you want to keep a package on a specific version, you may instead prevent upgrades of that specific package until you are ready to upgrade it, without affecting other packages that the repository provides.
Note that it is normally unnecessary to remove a line of the sources.list since you can comment it out (# at the start of the line), which prevents you from forgetting from what repository you installed a specific package.
PPA's maintainers are known for not following the packaging guide thoughtfully which may cause problems. If you are able to get the package through Ubuntu's own repositories, you are less risky of running into problems.