Kernel panic output image

I have a very specific and strange problem and ita kinda urgent to solve it in a timely manner. Yesterday I encountered a kernel panic on my servers looking at server management logs apparently there was a power outage and UPS was out of service for maintenance booting to the default kernel: cannot open "scripts/zfs" no such file or directory kernel panic "not syncing attempted to kill init"

I was able to boot from recovery mode in grub to a previous kernel then after checking system journal I deleted the problem a malfunctioning virtual network card causing problems so I removed the ovswitch, did an apt update and fix broken packages and install the new available kernel. After reboot I still get the kernel panic, and because I updated grub and installed new kernel, the previous kernel which was working is no longer available so im no longer able to boot the system.

Reinstalling system is out of the question so I need help to repair the kernel.

this is what I've tried So far I booted a live os, imported the zfs pool, I can see the pool is mounted new directory appears in the root directory of the live system and with 'zfs list' command I can see that the root directory of the zfs pool is mounted to / but I cant read the data when I cd to the directory its empty I did "zpool import rpool" then 2 directories appeared rpool/DATA and rpool/ROOT/pve-1 but they are both empty. with lsblk I can see most of the guest VMs partitions but not main system's. also after importing the zfs pool I did "mdadm --assemble --scan" two additional partitions appeared and I dont know what they are ( the reason I assembled raid arrays after importing zfs is because before importing the rpool mdadm wont recognize any raid arrays)

I believe I need to chroot to the storage with live os and repair zfs scripts

its worth mentioning that I have selinux on permissive mode, I i believe this is what caused the problem isn the first place.

A clean install would be my last option.

So, Is there anything I should try before rebuilding?

And, if I must rebuild, Can someone provide guidance on migrating my data?

system configuration: software RAID 5 filesystem ZFS storage : rpool zfs / rpool.

  • This somewhat related to the question, but I don't understand why you're mixing ZFS with mdadm. This seems to complicate things in an unnecessary way. Why not just stick with ZFS? Dec 3, 2022 at 14:37
  • Also, enabling SELinux on Ubuntu is (in my opinion) considered an advanced usecase (since this will break snaps for instance), so I'm not sure what other consequences this could have - maybe you discovered one of them. Dec 3, 2022 at 14:40
  • about the comment on mdadm its because I have software raid 5 thats why I believe some partitions not mounted. Dec 5, 2022 at 12:04


You must log in to answer this question.

Browse other questions tagged .