Change mySQL datadir: problems with apparmor

Question

I'm trying to change mysql data directory to a zfs dataset.

I've changed mysql.cnf and /etc/apparmor.d/usr.sbin.mysqld, but still have errors. I get the errors even when I try to stop apparmor service with systemctl stop apparmor.service. I have also tried creating a link to /etc/apparmor.d/usr.sbin.mysqld in /etc/apparmor.d/disable.

I've also tried this (don't understand why it should work, but tried anyway to avoid my question being marked as duplicated) and it failed.

It seems apparmor is not obeying me anymore. Am I doing something wrong, or it's something related with the Singularity?

This is my exit of apparmor_status after systemctl stop apparmor.service

15 profiles are loaded.
15 profiles are in enforce mode.
   /sbin/dhclient
   /usr/bin/lxc-start
   /usr/lib/NetworkManager/nm-dhcp-client.action
   /usr/lib/NetworkManager/nm-dhcp-helper
   /usr/lib/connman/scripts/dhclient-script
   /usr/lib/lxd/lxd-bridge-proxy
   /usr/lib/snapd/snap-confine
   /usr/lib/snapd/snap-confine//mount-namespace-capture-helper
   /usr/lib/snapd/snap-confine//snap_update_ns
   /usr/sbin/mysqld
   /usr/sbin/tcpdump
   lxc-container-default
   lxc-container-default-cgns
   lxc-container-default-with-mounting
   lxc-container-default-with-nesting
0 profiles are in complain mode.
0 processes have profiles defined.
0 processes are in enforce mode.
0 processes are in complain mode.
0 processes are unconfined but have a profile defined.

These are the error in journalctl -xe after a failed attempt to start mysql:

ene 06 12:56:32 mantra audit[3967]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3967/status" pid=3967 comm="mysqld" requested_mask="r" denied_mask="r" fsuid=114 ouid=114
ene 06 12:56:32 mantra audit[3967]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" pid=3967 comm="mysqld" requested_mask="r" denied_mask="r" fsuid=114 ouid=0
ene 06 12:56:32 mantra audit[3967]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3967/status" pid=3967 comm="mysqld" requested_mask="r" denied_mask="r" fsuid=114 ouid=114
ene 06 12:56:32 mantra kernel: audit: type=1400 audit(1515239792.499:95): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3967/status" pid=3967 comm="mysqld" requested_mask="r" denied_m
ene 06 12:56:32 mantra kernel: audit: type=1400 audit(1515239792.499:96): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/sys/devices/system/node/" pid=3967 comm="mysqld" requested_mask="r" 
ene 06 12:56:32 mantra kernel: audit: type=1400 audit(1515239792.499:97): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/proc/3967/status" pid=3967 comm="mysqld" requested_mask="r" denied_m
ene 06 12:56:32 mantra kernel: audit: type=1400 audit(1515239792.675:98): apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/pit/mysql/mantra.lower-test" pid=3967 comm="mysqld" requested_mask=
ene 06 12:56:32 mantra audit[3967]: AVC apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/pit/mysql/mantra.lower-test" pid=3967 comm="mysqld" requested_mask="c" denied_mask="c" fsuid=114 ouid
ene 06 12:56:32 mantra audit[3967]: AVC apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/pit/mysql/mantra.lower-test" pid=3967 comm="mysqld" requested_mask="c" denied_mask="c" fsuid=114 ouid
ene 06 12:56:32 mantra audit[3967]: AVC apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/pit/mysql/mantra.lower-test" pid=3967 comm="mysqld" requested_mask="c" denied_mask="c" fsuid=114 ouid
ene 06 12:56:32 mantra kernel: audit: type=1400 audit(1515239792.683:99): apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/pit/mysql/mantra.lower-test" pid=3967 comm="mysqld" requested_mask=
ene 06 12:56:32 mantra kernel: audit: type=1400 audit(1515239792.683:100): apparmor="DENIED" operation="mknod" profile="/usr/sbin/mysqld" name="/pit/mysql/mantra.lower-test" pid=3967 comm="mysqld" requested_mask
ene 06 12:56:32 mantra audit[3967]: AVC apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/pit/mysql/ibdata1" pid=3967 comm="mysqld" requested_mask="wr" denied_mask="wr" fsuid=114 ouid=114
ene 06 12:56:32 mantra kernel: audit: type=1400 audit(1515239792.715:101): apparmor="DENIED" operation="open" profile="/usr/sbin/mysqld" name="/pit/mysql/ibdata1" pid=3967 comm="mysqld" requested_mask="wr" denie
ene 06 12:56:33 mantra systemd[1]: mysql.service: Main process exited, code=exited, status=1/FAILURE

What could I try?

Answer 1

It's solved! I still don't know why apparmor was not stopped but I've been able to change datadir.

The problem was that I was creating a backup file of /etc/apparmor.d/usr.sbin.mysqld (ie, usr.sbin.mysql.backup) in the same directory and it was being processed after the file I was modifying, so the changes were overwritten. Big facepalm.

I found useful information in this post. After installing apparmor-utils and running aa-complain I got an error of "duplicated profile" or something like that and I was able to realize what the problem was.

I still got apparmor errors about /proc/NNN/status but I think they were there before the change. Everything is working fine now.