0

My Ubuntu 18 server has just this week started crashing every few days, at random interval the machine just hangs and then must be reset in VMWare. I have looked at the Kern.log file and Syslog file however i am unsure where i need to searching for to debug the crash.

In the leadup to the crash the syslog looks like this:

Mar 20 17:35:16 files kernel: [195514.936029] BUG: unable to handle kernel paging request at 0000000000390060
Mar 20 17:35:16 files kernel: [195514.938593] IP: rb_next+0x1a/0x50
Mar 20 17:35:16 files kernel: [195514.941535] PGD 0 P4D 0 
Mar 20 17:35:16 files kernel: [195514.944737] Oops: 0000 [#1] SMP NOPTI
Mar 20 17:35:16 files kernel: [195514.948324] Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm fscache input_leds edac_mce_amd joydev serio_raw vmw_balloon shpchp mac_hid vmw_vsock_vmci_transport vsock vmw_vmci sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear vmwgfx ttm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel drm_kms_helper syscopyarea pcbc sysfillrect sysimgblt fb_sys_fops drm hid_generic aesni_intel mptspi mptscsih usbhid mptbase aes_x86_64 crypto_simd glue_helper psmouse hid e1000 scsi_transport_spi cryptd i2c_piix4 pata_acpi floppy
Mar 20 17:35:16 files kernel: [195514.972324] CPU: 0 PID: 26770 Comm: kworker/0:2 Tainted: G             L   4.15.0-46-generic #49-Ubuntu
Mar 20 17:35:16 files kernel: [195514.978029] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Mar 20 17:35:16 files kernel: [195514.984719] Workqueue: cgroup_destroy css_free_work_fn
Mar 20 17:35:16 files kernel: [195514.987358] RIP: 0010:rb_next+0x1a/0x50
Mar 20 17:35:16 files kernel: [195514.990583] RSP: 0018:ffffbb48813abcc0 EFLAGS: 00010206
Mar 20 17:35:16 files kernel: [195514.993125] RAX: 0000000000390050 RBX: ffff97d278cd1220 RCX: ffff97d278cd1d61
Mar 20 17:35:16 files kernel: [195514.995624] RDX: 0000000000390050 RSI: ffff97d278cd1220 RDI: ffff97d2765c9de8
Mar 20 17:35:16 files kernel: [195514.998342] RBP: ffffbb48813abcc0 R08: ffff97d27ac00ff8 R09: ffff97d27fc22180
Mar 20 17:35:16 files kernel: [195515.001753] R10: 0000000000000040 R11: 0000000000000000 R12: ffff97d2765c9dd0
Mar 20 17:35:16 files kernel: [195515.005735] R13: ffff97d270127c00 R14: ffffffff90662320 R15: ffff97d27a5e29d0
Mar 20 17:35:16 files kernel: [195515.008846] FS:  0000000000000000(0000) GS:ffff97d27fc00000(0000) knlGS:0000000000000000
Mar 20 17:35:16 files kernel: [195515.043620] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 20 17:35:16 files kernel: [195515.044491] CR2: 0000000000390060 CR3: 0000000137e4e000 CR4: 00000000000406f0
Mar 20 17:35:16 files kernel: [195515.045235] Call Trace:
Mar 20 17:35:16 files kernel: [195515.046728]  kernfs_next_descendant_post+0x25/0xa0
Mar 20 17:35:16 files kernel: [195515.047502]  __kernfs_remove.part.12+0x57/0x1f0
Mar 20 17:35:16 files kernel: [195515.048197]  kernfs_remove+0x37/0x50
Mar 20 17:35:16 files kernel: [195515.048810]  sysfs_remove_dir+0x50/0x60
Mar 20 17:35:16 files kernel: [195515.049382]  kobject_del.part.5+0x13/0x40
Mar 20 17:35:16 files kernel: [195515.049976]  kobject_del+0xe/0x20
Mar 20 17:35:16 files kernel: [195515.051274]  sysfs_slab_unlink+0x1d/0x20
Mar 20 17:35:16 files kernel: [195515.052440]  shutdown_cache+0xbe/0x150
Mar 20 17:35:16 files kernel: [195515.053158]  memcg_destroy_kmem_caches+0x6a/0x90
Mar 20 17:35:16 files kernel: [195515.053830]  mem_cgroup_css_free+0x14c/0x190
Mar 20 17:35:16 files kernel: [195515.054531]  css_free_work_fn+0x4c/0x350
Mar 20 17:35:16 files kernel: [195515.055530]  process_one_work+0x1de/0x410
Mar 20 17:35:16 files kernel: [195515.056164]  worker_thread+0x32/0x410
Mar 20 17:35:16 files kernel: [195515.056735]  kthread+0x121/0x140
Mar 20 17:35:16 files kernel: [195515.057302]  ? process_one_work+0x410/0x410
Mar 20 17:35:16 files kernel: [195515.057911]  ? kthread_create_worker_on_cpu+0x70/0x70
Mar 20 17:35:16 files kernel: [195515.058534]  ? do_syscall_64+0x73/0x130
Mar 20 17:35:16 files kernel: [195515.059337]  ? SyS_exit_group+0x14/0x20
Mar 20 17:35:16 files kernel: [195515.059898]  ret_from_fork+0x22/0x40
Mar 20 17:35:16 files kernel: [195515.060446] Code: ff ff 0f 1f 44 00 00 66 2e 0f 1f 84 00 00 00 00 00 55 48 8b 0f 48 89 e5 48 39 cf 74 36 48 8b 47 08 48 85 c0 75 05 eb 1a 48 89 d0 <48> 8b 50 10 48 85 d2 75 f4 5d c3 48 3b 79 08 75 f8 48 8b 09 48 
Mar 20 17:35:16 files kernel: [195515.062141] RIP: rb_next+0x1a/0x50 RSP: ffffbb48813abcc0
Mar 20 17:35:16 files kernel: [195515.062681] CR2: 0000000000390060
Mar 20 17:35:16 files kernel: [195515.063127] ---[ end trace 17c26078e913f72a ]---
Mar 20 17:36:16 files systemd-udevd[464]: seq 10836 '/kernel/slab/:A-0005824/cgroup/task_struct(103827:phpsessionclean.service)' is taking a long time
Mar 20 17:38:16 files systemd-udevd[464]: seq 10836 '/kernel/slab/:A-0005824/cgroup/task_struct(103827:phpsessionclean.service)' killed
Mar 20 17:39:01 files CRON[29976]: (root) CMD (  [ -x /usr/lib/php/sessionclean ] && if [ ! -d /run/systemd/system ]; then /usr/lib/php/sessionclean; fi)
Mar 20 17:46:33 files kernel: [196192.229536] CIFS VFS: Free previous auth_key.response = 00000000f1fa7187
Mar 20 17:46:33 files kernel: [196192.229540] CIFS VFS: Free previous auth_key.response = 0000000032c978c7
Mar 20 17:46:33 files kernel: [196192.233023] CIFS VFS: Free previous auth_key.response = 0000000067c558e0
Mar 20 17:46:33 files kernel: [196192.543930] CIFS VFS: Free previous auth_key.response = 00000000502f9bd6
Mar 20 17:58:04 files kernel: [196883.751105] BUG: unable to handle kernel NULL pointer dereference at 0000000000000030
Mar 20 17:58:04 files kernel: [196883.753103] IP: anon_vma_fork+0xb0/0x130
Mar 20 17:58:04 files kernel: [196883.753542] PGD 0 P4D 0 
Mar 20 17:58:04 files kernel: [196883.753964] Oops: 0002 [#2] SMP NOPTI
Mar 20 17:58:04 files kernel: [196883.754373] Modules linked in: cmac arc4 md4 nls_utf8 cifs ccm fscache input_leds edac_mce_amd joydev serio_raw vmw_balloon shpchp mac_hid vmw_vsock_vmci_transport vsock vmw_vmci sch_fq_codel ib_iser rdma_cm iw_cm ib_cm ib_core iscsi_tcp libiscsi_tcp libiscsi scsi_transport_iscsi ip_tables x_tables autofs4 btrfs zstd_compress raid10 raid456 async_raid6_recov async_memcpy async_pq async_xor async_tx xor raid6_pq libcrc32c raid1 raid0 multipath linear vmwgfx ttm crct10dif_pclmul crc32_pclmul ghash_clmulni_intel drm_kms_helper syscopyarea pcbc sysfillrect sysimgblt fb_sys_fops drm hid_generic aesni_intel mptspi mptscsih usbhid mptbase aes_x86_64 crypto_simd glue_helper psmouse hid e1000 scsi_transport_spi cryptd i2c_piix4 pata_acpi floppy
Mar 20 17:58:04 files kernel: [196883.758578] CPU: 0 PID: 3360 Comm: mysqld Tainted: G      D      L   4.15.0-46-generic #49-Ubuntu
Mar 20 17:58:04 files kernel: [196883.759620] Hardware name: VMware, Inc. VMware Virtual Platform/440BX Desktop Reference Platform, BIOS 6.00 07/28/2017
Mar 20 17:58:04 files kernel: [196883.760740] RIP: 0010:anon_vma_fork+0xb0/0x130
Mar 20 17:58:04 files kernel: [196883.761301] RSP: 0018:ffffbb4881e77d70 EFLAGS: 00010286
Mar 20 17:58:04 files kernel: [196883.761864] RAX: 0000000000000000 RBX: ffff97d2781e11a0 RCX: 0000000000000502
Mar 20 17:58:04 files kernel: [196883.762439] RDX: ffff97d277f21d10 RSI: ffff97d277f22bc0 RDI: ffff97d27625cd80
Mar 20 17:58:04 files kernel: [196883.763018] RBP: ffffbb4881e77d98 R08: ffffdb487fc1e630 R09: 0000000000000000
Mar 20 17:58:04 files kernel: [196883.763610] R10: 00000007ff284bff R11: ffff97d277f224c0 R12: 0000000000000000
Mar 20 17:58:04 files kernel: [196883.764197] R13: ffff97d2769d4ea0 R14: ffff97d2790d2738 R15: ffff97d277f22d00
Mar 20 17:58:04 files kernel: [196883.764788] FS:  00007ff2a2fa7700(0000) GS:ffff97d27fc00000(0000) knlGS:0000000000000000
Mar 20 17:58:04 files kernel: [196883.765388] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
Mar 20 17:58:04 files kernel: [196883.765990] CR2: 0000000000000030 CR3: 0000000137e4e000 CR4: 00000000000406f0
Mar 20 17:58:04 files kernel: [196883.766637] Call Trace:
Mar 20 17:58:04 files kernel: [196883.767576]  copy_process.part.35+0xdb8/0x1b00
Mar 20 17:58:04 files kernel: [196883.768176]  _do_fork+0xdf/0x400
Mar 20 17:58:04 files kernel: [196883.769176]  ? __secure_computing+0x3f/0x100
Mar 20 17:58:04 files kernel: [196883.769784]  ? syscall_trace_enter+0xd7/0x2d0
Mar 20 17:58:04 files kernel: [196883.770344]  SyS_clone+0x19/0x20
Mar 20 17:58:04 files kernel: [196883.770923]  do_syscall_64+0x73/0x130
Mar 20 17:58:04 files kernel: [196883.771527]  entry_SYSCALL_64_after_hwframe+0x3d/0xa2
Mar 20 17:58:04 files kernel: [196883.772260] RIP: 0033:0x7ff2a4eb4b1c
Mar 20 17:58:04 files kernel: [196883.772854] RSP: 002bMar 20 19:53:39 files kernel: [    0.000000] Linux version 4.15.0-46-generic (buildd@lgw01-amd64-038) (gcc version 7.3.0 (Ubuntu 7.3.0-16ubuntu3)) #49-Ubuntu SMP Wed Feb 6 09:33:07 UTC 2019 (Ubuntu 4.15.0-46.49-generic 4.15.18)

The next line is 2 hours later when i've reset the machine.

Can any one give me some guidance on what i should be looking for here?

2
  • Can you please be specific about your Ubuntu. Are you using Ubuntu Core 18? or 18.04 LTS or 18.10? Standard releases use yy.mm format, whilst IoT & other releases use only yy (having only a single release per year)
    – guiverc
    Mar 20, 2019 at 22:23
  • The VMWare version might be helpful as well... Mar 20, 2019 at 23:26

0

You must log in to answer this question.

Browse other questions tagged .