Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I have a server running Xen 4.1 with Oneiric in the dom0 and each of the 4 domUs. The system disks of the domUs are LVM2 volumes built on top of an mdadm RAID1.

All the domU system disks are EXT4 and are created using snapshots of the same original template. 3 of them run perfectly, but one (called s-ub-02) keeps on being remounted read-only. A subsequent e2fsck results in a single "invalid extent" diagnosis:

e2fsck 1.41.14 (22-Dec-2010)
/dev/domu/s-ub-02-root contains a file system with errors, check forced.
Pass 1: Checking inodes, blocks, and sizes
Inode 525418 has an invalid extent
    (logical block 8959, invalid physical block 0, len 0)
Clear<y>? yes

Pass 2: Checking directory structure
Pass 3: Checking directory connectivity
Pass 4: Checking reference counts
Pass 5: Checking group summary information
/dev/domu/s-ub-02-root: 77757/655360 files (0.3% non-contiguous), 360592/2621440 blocks

The console shows typically the following errors for the system disk (xvda2):

[101980.903416] EXT4-fs error (device xvda2): ext4_ext_find_extent:732: inode #525418: comm apt-get: bad header/extent: invalid extent entries - magic f30a, entries 12, max 340(340), depth 0(0)
[101980.903473] EXT4-fs (xvda2): Remounting filesystem read-only

I have created new versions of the system disk. The same thing always happens. This, and the fact that the disk is ultimately on a RAID1, leads me to preclude a hardware disk error.

The only obvious distinguishing feature of this domU is the presence of nfs-kernel-server, so I suspect that. Its exports file looks like this:

/exports/users           192.168.0.0/255.255.248.0(rw,sync,no_subtree_check)
/exports/media/music     192.168.0.0/255.255.248.0(rw,sync,no_subtree_check)
/exports/media/pictures  192.168.0.0/255.255.248.0(rw,sync,no_subtree_check)
/exports/opt             192.168.0.0/255.255.248.0(rw,sync,no_subtree_check)

/exports/users and /exports/opt are LVM2 volumes from the same volume group as the system disk. /exports/media is an EXT2 volume. (There is an issue where clients see /exports/media/pictures as being a read-only volume, which I mention for completeness.)

With the exception of the read-only problem, the NFS server appears to work correctly under light load for several hours before the "invalid extent" problem occurs.

There are no helpful entries in /var/log. All of a sudden, no more files are written, so you can see when the disk was remounted read-only, but there is no indication of what the cause might be.

Can anyone help me with this problem?

Steve

share|improve this question
    
I'm unclear as to what domain is managing what devices. Do you mean that dom0 is running mdadm on top of the real disks, or that the dumUs are running mdadm on top of virtual disks? –  psusi Jan 4 '12 at 14:41
    
The dom0 is running mdadm (a simple RAID1 with 2 465GB disks) as /dev/md0. LVM2 on the dom0 uses /dev/md0 as a physical volume to create a volume group called "domu". The system disks are created in domu and passed to the domUs as /dev/xvdaN, etc. –  Stephen Winnall Jan 4 '12 at 15:28
    
I've taken a closer look at the inodes that have been reported as having invalid extents. They correspond to /var/cache/apt/pkgcache.bin, /var/lib/dpkg/info/nfs-kernel-server.list, /var/cache/apt/srcpkgcache.bin, and /var/lib/apt/lists/security.ubuntu.com_ubuntu_dists_oneiric-security_universe_so‌​urce_Sources. Looks like it might be an apt-related problem. But apt is not causing any problems on the other domUs... –  Stephen Winnall Jan 4 '12 at 21:02
    
You aren't passing the same LV to multiple domUs are you? –  psusi Jan 4 '12 at 23:10
    
No, I'm not. Though I did try it once. It would be cool if you could do that with read-only partitions. But I digress. –  Stephen Winnall Jan 6 '12 at 11:34
show 5 more comments

1 Answer 1

This problem was solved by the OP:

I finally solved this problem by ditching ext4 and going back to ext3 after ext4 started playing up again. There's something weird going on in that particular VM, but I can't spend any more time on trying to find out what it is.

share|improve this answer
add comment

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.