It is probably safe to remove Apport, but doing so is completely unnecessary for protecting your privacy.
Apport's core dump interception is disabled by default on stable versions of Ubuntu. To completely disable Apport, edit
1 means it's enabled.
0 means it's disabled.) Then reboot (or manually stop the service) to apply the change. See the Apport documentation, and especially the section on current versions of Ubuntu, for more details.
Apport does not use
dconf for setting whether or not it is enabled. It is a system service, and whether or not it runs (i.e., whether or not it intercepts crashes, creates crash dumps, and makes it possible for the user to submit them to Launchpad or other web services) is not determined by any user's per-user settings.
If you do remove Apport, you will lose the ability to use it to report non-crash bugs (with Help > Report a Bug) or running
ubuntu-bug. If then you or another administrator decides to reinstall it for this purpose, then depending on how you removed it and what system you are running when you reinstall it, it might then become enabled for intercepting crashes and creating/submitting core dumps.
So if you're vigilant (or even if you're not), disabling Apport should be quite sufficient. If you're not vigilant, then uninstalling Apport might actually make it more likely to intercept and submit crash data in the future (as described above), if you reinstall it for its other functionality and forget to check if the crash reporting service is enabled.
If you do decide to remove Apport, you can check first to see if doing so would break everything, by simulating its removal:
apt-get -s remove apport
That will show you what other packages would be removed as a consequence. Then, to actually remove it:
sudo apt-get remove apport
Please note that you probably do not want to
purge it, since if you remove it you'll want to keep the
apport configuration file in
/etc/default that has
enabled=0. (Just know that this doesn't necessarily guarantee that the crash reporting service will never be enabled upon reinstallation, in any future version.)