41

It slows my system boot time.

Can i disable it?

What will happen if i disable it on boot?

I am using Ubuntu version 18.04.

2 Answers 2

43

The systemd-journal-flush.service asks the journal daemon to flush any log data stored in /run/log/journal into /var/log/journal, if persistent storage is enabled. In case you have (already) huge log files, this will result in slower booting. Further, the disk (with /var/log) has to be mounted in a writable modus to do so.

To sum it up: huge old log files, which are checked during boot and the appending of new log data results in slower boot time.

To check the journalctl log size type

journalctl --disk-usage

In order to get the time and disk space information of flush processing, enter the following command

journalctl -b --unit systemd-journald

The corresponding output will look like

-- Logs begin at Sat 2018-12-08 00:40:23 CET, end at Mon 2018-12-10 19:40:27 CET. --
Dec 10 12:51:38 ubuntu01 systemd-journald[479]: Journal started
Dec 10 12:51:38 ubuntu01 systemd-journald[479]: Runtime journal (/run/log/journal/265c93c062bf4c8da41abfe2ae793452) is 4.7M, max 38.3M, 33.5M free.
Dec 10 12:51:38 ubuntu01 systemd-journald[479]: Time spent on flushing to /var is 7.066904s for 132 entries.
Dec 10 12:51:38 ubuntu01 systemd-journald[479]: System journal (/var/log/journal/265c93c062bf4c8da41abfe2ae793452) is 128.0M, max 256.0M, 128M free.

You can either

  • Disable the service (not recommended)

    Then it is possible that not all log data is written to disk; annoying when debugging boot faults. Journald is a fundamental service in systemd Linux and many other services depends on it.

  • Verify the journal file for internal consistency:

      journalctl --verify
    

    Look out that every journal shows up PASS.

  • Use a journalctl --vacuum command

    From journalctl -h

    --vacuum-size=BYTES Reduce disk usage below specified size
    --vacuum-files=INT Leave only the specified number of journal files
    --vacuum-time=TIME Remove journal files older than specified time

    Hence do a

      sudo journalctl --vacuum-size=1G --vacuum-time=5d --vacuum-files=5
    
  • Change storage type of systemd-journal-flush.service

    First check your storage type with:

         systemctl cat systemd-journal-flush.service  | grep -i storage
    

    From man journald.conf

    Storage=

    Controls where to store journal data. One of "volatile", "persistent", "auto" and "none".

    If "volatile", journal log data will be stored only in memory, i.e. below the /run/log/journal hierarchy (which is created if needed).

    If "persistent", data will be stored preferably on disk, i.e. below the /var/log/journal hierarchy (which is created if needed), with a fallback to /run/log/journal (which is created if needed), during early boot and if the disk is not writable.

    "auto" is similar to "persistent" but the directory /var/log/journal is not created if needed, so that its existence controls where log data goes.

    "none" turns off all storage, all log data received will be dropped. Forwarding to other targets, such as the console, the kernel log buffer, or a syslog socket will still work however. Defaults to "auto".

    Edit the file

    sudo nano /etc/systemd/journald.conf

    In the journal section uncomment and alter:

    Storage=auto SystemMaxUse=200M SystemKeepFree=1G
    SystemMaxFileSize=1G SystemMaxFiles=5

Recommendation

I recommend limiting the size of the SystemMaxFileSize key to 50MB -->

SystemMaxFileSize=50M

Finally, in case your Ubuntu is not running on an important server, I suggest changing the data storage to volatile:

Storage=volatile

SystemMaxUse control how much disk space the journal may use up at most.

SystemKeepFree and RuntimeKeepFree control how much disk space systemd-journald shall leave free for other uses. systemd-journald will respect both limits and use the smaller of the two values.

If you run the boot in debug mode and trace the system calls (strace) you may found out that the flush writing has a very poor i/o performance. In my case it was unclear why. Maybe some kernel messages spam the log file (note that after 10000 messages the unit is blocked per default, but journald has to manage this, which maybe cause the poor performance). In that case step over the messages and search for errors, which haven't necessarily marked as errors.

journalctl -b --output short-monotonic

and

journalctl -b -p 1..4 --output short-monotonic

The --output short-monotonicflag prints the time steps in contrast to the default UTC time.

Finally, remove the old log files by

sudo rm -rf /var/log/journal

Save & reboot.

10
  • I don't see why huge log file would slow the boot ? it flushes the log in /run/log/journal but that's only the log since the last boot
    – solsTiCe
    Nov 24, 2018 at 16:23
  • 1
    I'am not absolute sure. It could also be the (useless?) dependency of systemd-user-sessions-flush.service on systemd-user-sessions.service (seehttps://github.com/systemd/systemd/pull/10502). However, I experienced that limiting the log file to less than 1GB speeds up boot time. I read through the systemd/journal/* code, but didn't found sth interesting.
    – abu_bua
    Nov 24, 2018 at 19:15
  • 1
    Maybe due to de-/compression (lz4)? Further log data doesn't simply get streamd into the file; it gets journaled and a hash table for faster searching modified, objects, ...
    – abu_bua
    Nov 24, 2018 at 19:37
  • 1
    I think a comment on how the journal files and /var/log files relate might be useful; I'm not clear on that but it seems to me that the journal is active logging and when it's flushed that is writing the active data to the disk in the normal /var/log files?
    – pbhj
    Dec 10, 2018 at 14:40
  • 2
    its definitely slowing things down. seems like insufficient testing is done on this by the developers to deliver safe defaults. Mar 19, 2023 at 17:46
4

According to this post from the systemd developer homepage, you can fix it by changing the Unit file.

To do so, open /lib/systemd/system/systemd-journal-flush.service, eg

sudo vim /lib/systemd/system/systemd-journal-flush.service

and change the Before dependency from

 Before=systemd-user-sessions.service systemd-tmpfiles-setup.service

to

 Before=systemd-tmpfiles-setup.service

This fix will be automatically altered for systemd versions > v240.

Don't forget to save the file.

You must log in to answer this question.

Not the answer you're looking for? Browse other questions tagged .