28

I am running MariaDB 10.0.23-0 on Ubuntu 15.10 as a LAMP server. Running sudo /etc/init.d/mysql start results in:

Job for mariadb.service failed because a timeout was exceeded. See "systemctl status mariadb.service" and "journalctl -xe" for details.

The output of systemctl status mariadb.service is:

● mariadb.service - MariaDB database server
   Loaded: loaded (/lib/systemd/system/mariadb.service; enabled; vendor preset: enabled)
  Drop-In: /etc/systemd/system/mariadb.service.d
           └─migrated-from-my.cnf-settings.conf
   Active: failed (Result: timeout) since Sat 2016-03-26 22:52:42 EDT; 26s ago
  Process: 8707 ExecStart=/usr/sbin/mysqld $MYSQLD_OPTS $_WSREP_NEW_CLUSTER (code=exited, status=0/SUCCESS)
  Process: 8706 ExecStartPre=/usr/bin/install -m 755 -o mysql -g root -d /var/run/mysqld (code=exited, status=0/SUCCESS)
 Main PID: 8707 (code=exited, status=0/SUCCESS)

Mar 26 22:52:39 boggan systemd[1]: mariadb.service: Start operation timed out. Terminating.
Mar 26 22:52:39 boggan mysqld[8707]: 2016-03-26 22:52:39 140105856617216 [Note] /usr/sbin/mysqld: Normal shutdown
Mar 26 22:52:39 boggan mysqld[8707]: 2016-03-26 22:52:39 140105856617216 [Note] Event Scheduler: Purging the queue. 0 events
Mar 26 22:52:39 boggan mysqld[8707]: 2016-03-26 22:52:39 140104920164096 [Note] InnoDB: FTS optimize thread exiting.
Mar 26 22:52:39 boggan mysqld[8707]: 2016-03-26 22:52:39 140105856617216 [Note] InnoDB: Starting shutdown...
Mar 26 22:52:42 boggan mysqld[8707]: 2016-03-26 22:52:42 140105856617216 [Note] InnoDB: Shutdown completed; log sequence number 3336953
Mar 26 22:52:42 boggan mysqld[8707]: 2016-03-26 22:52:42 140105856617216 [Note] /usr/sbin/mysqld: Shutdown complete
Mar 26 22:52:42 boggan systemd[1]: Failed to start MariaDB database server.
Mar 26 22:52:42 boggan systemd[1]: mariadb.service: Unit entered failed state.
Mar 26 22:52:42 boggan systemd[1]: mariadb.service: Failed with result 'timeout'`

The first systemd line there is kind of a "well duh". I know it timed out. The second systemd, after the mysqld lines is a bit mystifying, because it does in fact start. An application (OwnCloud, specifically) that depends on the database works normally... for the minute-and-change that MariaDB is up.

Another question suggested using time /etc/init.d/mysql start to determine how long it was taking. I ran it repeatedly to confirm the time - it's a few seconds on either side of 90s each time.

Other research lead me to check file permissions, which are fine... besides, it does start up, temporarily. I've poked and prodded to the best of my (admittedly limited when it comes to Linux) ability, and I haven't made any headway.

So, the question is... How do I get the MariaDB service to stay up?

As an extra wrinkle, after writing this question, I left the machine up and running. I came back to it a week later (I didn't touch it between). Using the exact same command, sudo /etc/init.d/mysql start, was successful. The mysql daemon started and ran; it came back with a an [ ok ] report. I rebooted for experimentation's sake, and I'm back where I started.

In case it matters, the output of journalctl -xe is:

Apr 02 23:51:44 boggan systemd[1]: Stopped Read required files in advance.
-- Subject: Unit ureadahead.service has finished shutting down
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit ureadahead.service has finished shutting down.
Apr 02 23:51:55 boggan mysqld[2645]: 2016-04-02 23:51:55 140386161068800 [Note] InnoDB: Online DDL : Start
Apr 02 23:51:55 boggan mysqld[2645]: 2016-04-02 23:51:55 140386161068800 [Note] InnoDB: Online DDL : Start reading clustered index of the table and create temporary files
Apr 02 23:51:55 boggan mysqld[2645]: 2016-04-02 23:51:55 140386161068800 [Note] InnoDB: Online DDL : End of reading clustered index of the table and create temporary files
Apr 02 23:51:55 boggan mysqld[2645]: 2016-04-02 23:51:55 140386161068800 [Note] InnoDB: Online DDL : Completed
Apr 02 23:51:55 boggan mysqld[2645]: 2016-04-02 23:51:55 140386161068800 [Note] InnoDB: Online DDL : Completed
Apr 02 23:52:06 boggan dbus[713]: [system] Failed to activate service 'org.bluez': timed out
Apr 02 23:52:37 boggan systemd[1]: mariadb.service: Start operation timed out. Terminating.
Apr 02 23:52:37 boggan mysqld[2645]: 2016-04-02 23:52:37 140386097400576 [Note] /usr/sbin/mysqld: Normal shutdown
Apr 02 23:52:37 boggan kernel: audit: type=1400 audit(1459655557.935:31): apparmor="DENIED" operation="sendmsg" profile="/usr/sbin/mysqld" name="/run/systemd/notify" pid=2645 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=122 ouid=0
Apr 02 23:52:37 boggan audit[2645]: AVC apparmor="DENIED" operation="sendmsg" profile="/usr/sbin/mysqld" name="/run/systemd/notify" pid=2645 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=122 ouid=0
Apr 02 23:52:37 boggan mysqld[2645]: 2016-04-02 23:52:37 140386097400576 [Note] Event Scheduler: Purging the queue. 0 events
Apr 02 23:52:37 boggan mysqld[2645]: 2016-04-02 23:52:37 140385225500416 [Note] InnoDB: FTS optimize thread exiting.
Apr 02 23:52:37 boggan mysqld[2645]: 2016-04-02 23:52:37 140386097400576 [Note] InnoDB: Starting shutdown...
Apr 02 23:52:39 boggan mysqld[2645]: 2016-04-02 23:52:39 140386097400576 [Note] InnoDB: Shutdown completed; log sequence number 3360838
Apr 02 23:52:39 boggan mysqld[2645]: 2016-04-02 23:52:39 140386097400576 [Note] /usr/sbin/mysqld: Shutdown complete
Apr 02 23:52:39 boggan kernel: audit: type=1400 audit(1459655559.419:32): apparmor="DENIED" operation="sendmsg" profile="/usr/sbin/mysqld" name="/run/systemd/notify" pid=2877 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=122 ouid=0
Apr 02 23:52:39 boggan audit[2877]: AVC apparmor="DENIED" operation="sendmsg" profile="/usr/sbin/mysqld" name="/run/systemd/notify" pid=2877 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=122 ouid=0
Apr 02 23:52:39 boggan audit[2645]: AVC apparmor="DENIED" operation="sendmsg" profile="/usr/sbin/mysqld" name="/run/systemd/notify" pid=2645 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=122 ouid=0
Apr 02 23:52:39 boggan kernel: audit: type=1400 audit(1459655559.419:33): apparmor="DENIED" operation="sendmsg" profile="/usr/sbin/mysqld" name="/run/systemd/notify" pid=2645 comm="mysqld" requested_mask="w" denied_mask="w" fsuid=122 ouid=0
Apr 02 23:52:39 boggan systemd[1]: Failed to start MariaDB database server.
-- Subject: Unit mariadb.service has failed
-- Defined-By: systemd
-- Support: http://lists.freedesktop.org/mailman/listinfo/systemd-devel
-- 
-- Unit mariadb.service has failed.
-- 
-- The result is failed.
Apr 02 23:52:39 boggan systemd[1]: mariadb.service: Unit entered failed state.
Apr 02 23:52:39 boggan systemd[1]: mariadb.service: Failed with result 'timeout'.
Improve this question
4
  • 2
    The journalctl -xe output is truncated, can you update this? Have a closer look at the apparmor="DENIED" messages (if apparmor is activated on your OS) as this could be an issue during mariadb start.
    – tlo
    Commented Apr 1, 2016 at 13:43
  • @tlo I shall... it will just have to wait until this evening. I don't have access to the machine from where I am. After all, I couldn't get it to function when I was sitting at it, so why bother setting it up for remote access. I'll definitely look into apparmor, too. If it was activated, it was activated by default. I haven't changed anything installed by the system, just added the LAMP stuff.
    – T.J.L.
    Commented Apr 1, 2016 at 13:52
  • @tlo Updated output, and added a bit of a wrinkle to the description. Instead of banging on it, I'm gonna walk away for an hour or two, and see what happens...
    – T.J.L.
    Commented Apr 3, 2016 at 4:01
  • 1
    @tlo Thank you for the help. apparmor was the culprit.
    – T.J.L.
    Commented Apr 3, 2016 at 7:06

4 Answers 4

Reset to default
31

I had quite the same issue after upgrading from mysql to mariadb. The strange thing was that service mariadb start failed on timeout (either at system boot or manualy) whereas service mysql start succeded.

The explanation given by T.J.L. is right but the correction didn't work for me.

$ sudo aa-complain /usr/sbin/mysqld
Setting /usr/sbin/mysqld to complain mode.

ERROR: /etc/apparmor.d/usr.sbin.mysqld contains no profile

So I disabled the profile (with aa-disable which seems to be equivalent to plutocrat's solution)

$ sudo aa-disable /usr/sbin/mysqld
Disabling /usr/sbin/mysqld.

I disabled mysqld-akonadi and mysqld-digikam as well.

An apparmor reload was not enough, so I had to reboot and mariadb started perfectly well.

Improve this answer
2
  • Confirming that couldn't find a way to make it work without rebooting.
    – Meetai.com
    Commented Feb 14, 2019 at 6:30
  • This answer worked for me on Kubuntu 18.04.2 LTS. complain and ... apparmor reload (answer T.J.L.) was indeed not enough.
    – Marten Koetsier
    Commented Mar 17, 2019 at 21:26
27

apparmor was the culprit. Despite the contents of /etc/apparmor.d/usr.sbin.mysqld being nothing but comments and claiming that it was there so that apparmor wouldn't choke on MariaDB, that's exactly what was happening.

AppArmor and MySQL on an Oracle blog provided what I needed to figure out what was going on.

sudo aa-status shows you what apparmor is doing; what actually has an enforced policy, versus what's just set to complain.

sudo apt-get install apparmor-utils adds a few commands that make the apparmor profiles easier to deal with, such as...

sudo aa-complain /usr/sbin/mysqld turns the profile from "enforce" to complain. (aa-enforce turns it back.)

Once that's done, sudo service apparmor reload restarts apparmor, and voila... sudo /etc/init.d/mysql start works, and the server stays up.

Improve this answer
3
  • 1
    Same for me. I upgraded from Ubuntu 14.04 to 16.04 and lost the ability to run MySQL. Now it works! Thanks so much for detailing this :D. I've been looking for weeks!
    – Sawtaytoes
    Commented Apr 25, 2017 at 7:33
  • Doesn't quite do it for me, but thanks for the tip with apparmor-utils. Three years later I'm getting ERROR: /etc/apparmor.d/usr.sbin.mysqld contains no profile.
    – YetiCGN
    Commented Jun 27, 2019 at 16:55
  • didn't work for me either - I get ERROR: /etc/apparmor.d/usr.sbin.mysqld doesn't contain a valid profile for /usr/sbin/mysqld (syntax error?) when trying to do the aa-complain command. this is a clean installation on a system that has never had mysql or mariadb on before - how does it get to be soo complicated???
    – Andy
    Commented Jul 7, 2021 at 8:58
18

I had to completely disable mysql in apparmor. An aa-complain wouldn't do anything for me. So ...

ln -s /etc/apparmor.d/usr.sbin.mysqld /etc/apparmor.d/disable/

Then reboot

Improve this answer
2
  • Thank you! This was the only solution to my problem! I also upgraded from mysql to mariadb
    – Thomas Gatt
    Commented Feb 10, 2018 at 8:03
  • this was what worked for me as well, thank you very much
    – Eman
    Commented Jul 24, 2019 at 3:50
12

A simple solution is to remove any unknown AppArmor profiles:

aa-remove-unknown
Removing '/snap/core/6350/usr/lib/snapd/snap-confine'
Removing '/usr/sbin/mysqld'

It works!

Improve this answer
5
  • 2
    This was actually what I needed to do to get things running, so thanks. The accepted answer above would give me ERROR: /etc/apparmor.d/usr.sbin.mysqld contains no profile as that's exactly true, given the file only consists of comments. Maybe in a newer version of AppArmor they set it to fail with those files, while it worked in 2016.
    – YetiCGN
    Commented Jun 27, 2019 at 16:53
  • 2
    This is the correct answer (at least in 2019). What happens is that after deinstallation of MySql, the AppArmor profile for /usr/sbin/mysqld is still loaded in the kernel. Running aa-remove-unknown (or rebooting) solves this.
    – zwets
    Commented Oct 8, 2019 at 21:26
  • This is the real solution. Other answers didn't work for me
    – Tofandel
    Commented Feb 24, 2021 at 0:05
  • This got rid of the apparmor errors but the database still wouldn't start - systemctl status mariadb.service and journalctl -xe just said the service exited but no error details. Eventually I found a log file called /var/log/mysql/error.log with a line saying InnoDB: Invalid flags 0x4800 in ./ibdata1. uninstalling and reinstalling didn't help but what did eventually work was uninstalling, delete the /var/lib/mysql directory, reboot, and re-install
    – Andy
    Commented Jul 7, 2021 at 15:34
  • 1
    I had it the same as Andy, but i solved it a bit more elegantly. I did aa-remove-unknown to get from app armour errors to nondescript errors. I uninstalled mariadb packets sudo apt remove mariadb-server mariadb-client. Then, instead of removing /var/lib/mysql, I did sudo apt remove mysql-common --purge and then reinstalled mariadl-server and mariadb-client. No reboot required and sudo systemctl start mariadb.service worked. My problem (in addition to the app armour) was that I had mysql installed on a system for just a second before I installed MariaDB.
    – Matjaz
    Commented Aug 11, 2021 at 20:20

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .