How to receive a desktop notification on a kernel lockup or bug/warning?

Question

Apport doesn't react at all on any segfaults/problems warnings that happen in kernel [modules], the only way I know of to see them is to check kernel log (dmesg). I'd like to receive a notification immediately if this happens, is it possible?

Examples

https://bugs.launchpad.net/ubuntu/+source/linux/+bug/1510723

[17348.408329] ------------[ cut here ]------------
[17348.408353] WARNING: CPU: 0 PID: 12897 at drivers/gpu/drm/i915/intel_display.c:11098 intel_check_page_flip+0x107/0x120 [i915]()
[17348.408355] Kicking stuck page flip: queued at 1040374, now 1040379
[17348.408356] Modules linked in: ...
[17348.408430] CPU: 0 PID: 12897 Comm: gdb Not tainted 4.2.0-0.rc6.git0.2.fc23.x86_64+debug #1
[17348.408432] Hardware name: Gigabyte Technology Co., Ltd. Z87M-D3H/Z87M-D3H, BIOS F11 08/12/2014
[17348.408433]  0000000000000000 0000000055ba9279 ffff8807fdc03ce8 ffffffff81868d8e
[17348.408436]  0000000000000000 ffff8807fdc03d40 ffff8807fdc03d28 ffffffff810ab406
[17348.408438]  ffff8807fdc03d58 ffff8807f32e7000 ffff8807f29c5000 0000000000000000
[17348.408441] Call Trace:
[17348.408442]  <IRQ>  [<ffffffff81868d8e>] dump_stack+0x4c/0x65
[17348.408451]  [<ffffffff810ab406>] warn_slowpath_common+0x86/0xc0
[17348.408453]  [<ffffffff810ab495>] warn_slowpath_fmt+0x55/0x70
[17348.408466]  [<ffffffffa0166917>] intel_check_page_flip+0x107/0x120 [i915]
[17348.408477]  [<ffffffffa012c028>] ironlake_irq_handler+0x288/0xbf0 [i915]
[17348.408480]  [<ffffffff81026d79>] ? sched_clock+0x9/0x10
[17348.408483]  [<ffffffff810e6ddc>] ? local_clock+0x1c/0x20
[17348.408486]  [<ffffffff8111f0d0>] handle_irq_event_percpu+0x40/0x490
[17348.408488]  [<ffffffff8111f564>] handle_irq_event+0x44/0x70
[17348.408491]  [<ffffffff811228f8>] handle_edge_irq+0x78/0x140
[17348.408493]  [<ffffffff8101f2e2>] handle_irq+0x72/0x120
[17348.408496]  [<ffffffff8112d18e>] ? rcu_irq_enter+0x8e/0xa0
[17348.408499]  [<ffffffff8187575f>] do_IRQ+0x4f/0xe0
[17348.408502]  [<ffffffff818733b0>] common_interrupt+0x70/0x70
[17348.408503]  <EOI>  [<ffffffff8127c72f>] ? unlazy_walk+0x13f/0x310
[17348.408508]  [<ffffffff8127c6fd>] ? unlazy_walk+0x10d/0x310
[17348.408510]  [<ffffffff8127d163>] lookup_fast+0x293/0x3b0
[17348.408513]  [<ffffffff813969a5>] ? security_inode_permission+0x45/0x70
[17348.408515]  [<ffffffff8127d5b9>] walk_component+0x49/0x2a0
[17348.408517]  [<ffffffff8127e2f2>] ? path_init+0x552/0x7f0
[17348.408519]  [<ffffffff8127e2aa>] ? path_init+0x50a/0x7f0
[17348.408521]  [<ffffffff8127e66d>] path_lookupat+0x5d/0x110
[17348.408523]  [<ffffffff81280219>] filename_lookup+0xa9/0x180
[17348.408526]  [<ffffffff8124bd0e>] ? kmem_cache_alloc+0x33e/0x360
[17348.408527]  [<ffffffff8127fe26>] ? getname_flags+0x56/0x1f0
[17348.408531]  [<ffffffff812943e5>] ? mntput_no_expire+0x5/0x3c0
[17348.408533]  [<ffffffff812803ca>] user_path_at_empty+0x3a/0x50
[17348.408536]  [<ffffffff81274476>] vfs_fstatat+0x66/0xc0
[17348.408539]  [<ffffffff81274a0e>] SYSC_newstat+0x2e/0x60
[17348.408542]  [<ffffffff81003044>] ? lockdep_sys_exit_thunk+0x12/0x14
[17348.408545]  [<ffffffff81274b9e>] SyS_newstat+0xe/0x10
[17348.408547]  [<ffffffff8187282e>] entry_SYSCALL_64_fastpath+0x12/0x76
[17348.408550] ---[ end trace f4b96237241a9872 ]---

Unreported, trying to reproduce it reliably, thus the question:

[39204.778745] NMI watchdog: BUG: soft lockup - CPU#1 stuck for 23s! [thumbnail.so:11253]
[39204.778748] Modules linked in: ...
[39204.778795] CPU: 1 PID: 11253 Comm: thumbnail.so Tainted: P        W  OEL  4.2.0-23-generic #28-Ubuntu
[39204.778796] Hardware name: LENOVO 20208/20208  , BIOS H1ET85WW(1.23) 04/15/2014
[39204.778798] task: ffff8801f95c2940 ti: ffff8801dee20000 task.ti: ffff8801dee20000
[39204.778800] RIP: 0010:[<ffffffff810c422e>]  [<ffffffff810c422e>] native_queued_spin_lock_slowpath+0x15e/0x170
[39204.778806] RSP: 0018:ffff8801dee23e48  EFLAGS: 00000202
[39204.778807] RAX: 0000000000000101 RBX: ffff8800884bc480 RCX: 0000000000000001
[39204.778809] RDX: 0000000000000101 RSI: 0000000000000001 RDI: ffff8801dee23f58
[39204.778810] RBP: ffff8801dee23e48 R08: 0000000000000101 R09: ffff8800b2891064
[39204.778811] R10: 0000000000000001 R11: ffff8801f94582c0 R12: 000000118120d84e
[39204.778812] R13: ffff8801dee23eb8 R14: ffff8801dee23db8 R15: ffff8801f9568b00
[39204.778814] FS:  00007faec8c43780(0000) GS:ffff88022e240000(0000) knlGS:0000000000000000
[39204.778815] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[39204.778816] CR2: 0000000000f5f000 CR3: 0000000189284000 CR4: 00000000001406e0
[39204.778817] Stack:
[39204.778819]  ffff8801dee23e58 ffffffff817efdbc ffff8801dee23e88 ffffffffc0662dec
[39204.778821]  ffff8801dee23e98 00000000c61800cc ffff8800b2891000 fffffffffffffffb
[39204.778823]  ffff8801dee23ea8 ffffffff81266b12 0000000000f0a8d8 ffff8801f9568b00
[39204.778825] Call Trace:
[39204.778831]  [<ffffffff817efdbc>] _raw_spin_lock_bh+0x2c/0x30
[39204.778862]  [<ffffffffc0662dec>] wl_proc_read+0x3c/0xc0 [wl]
[39204.778866]  [<ffffffff81266b12>] proc_reg_read+0x42/0x70
[39204.778869]  [<ffffffff811fccd8>] __vfs_read+0x18/0x40
[39204.778871]  [<ffffffff811fd2a7>] vfs_read+0x87/0x130
[39204.778873]  [<ffffffff811fe025>] SyS_read+0x55/0xc0
[39204.778877]  [<ffffffff817f02b2>] entry_SYSCALL_64_fastpath+0x16/0x75

Answer 1

These particular messages are WARNINGs and Soft Lockups. Sometimes you can continue running without issue and therefore aren't always show-stopping problems. Depending on what you want to do you have few options.

You can crashdump on soft lockups, panics, or other events. This wiki has relevant information on how to set this up: https://wiki.ubuntu.com/Kernel/CrashdumpRecipe Note, when you crashdump your computer will reset, so this may not be what you want.

Another option would be to poll for these events using a small bash script. Here is an attempt:

 #!/bin/bash -x                                                                  

rm /tmp/dmesg.*                                                                 
touch /tmp/dmesg.old                                                            
while true; do                                                                  
  dmesg > /tmp/dmesg.new                                                        
  diff /tmp/dmesg.old /tmp/dmesg.new > /tmp/dmesg.diff                          
  if egrep \(BUG\|WARNING\) /tmp/dmesg.diff; then                 
    notify-send "$(tail -n 3 /tmp/dmesg.diff)"                                  
  fi                                                                            
  dmesg > /tmp/dmesg.old                                                        
  sleep 10                                                                      
done     

This will show up as a notification when a new BUG or WARNING appears.