Take the 2-minute tour ×
Ask Ubuntu is a question and answer site for Ubuntu users and developers. It's 100% free, no registration required.

I want to intentionally damage a file in order to test the claims that btrfs can heal itself. The article talks about taking the filesystem off, damaging a photo by "flipping" a single bit and then remounting it. In older filesystems this would just be corrupted but it's supposed to fix itself in btrfs. In theory, this makes sense but I really want to test it.

The problem is the article doesn't explain how to do any of that.
How would I go about changing a single bit in a very specific part of a filesystem?

I should also point out that this must be done on an offline filesystem so that btrfs doesn't see my write as intentional.

Edit: While the question (and discussion) talks a lot about btrfs, I'd like to know if there are filesystem independant methods of implementing this sort of corruption (so that it can be compared across different RAID types/controllers/etc).

share|improve this question
    
@Dan I mean that if I edit the file directly, btrfs (or any filesystem for that matter) would count that as a valid write. It wouldn't give the corruption I'm looking for. –  Oli Jan 16 at 16:44
    
Is this a test filesystem (I.e., you don't care about the contents, or are perfectly fine with restoring from backup? Also, are you using a single btrfs partition on a single drive, or a single partition on top of a RAID array, or some other configuration? –  Darth Android Jan 16 at 16:45
1  
If btrfs supports the right ioctls (not sure if it does), you can use filefrag -v to find out exactly where a file is located. –  derobert Jan 16 at 16:51
3  
@Oli I suspect that you'd find a more interested audience on U&L, both in terms of votes and in terms of answers. plus, this. –  strugee Jan 16 at 19:11
1  
Shoot a cosmic ray in just the right spot. –  smcg Jan 16 at 20:50

5 Answers 5

I'm not an expert, but the btrfs-progs package actually includes a tool specifically to do this, although you may have to build from source. In any case, once you have installed or built btrfs-progs, you should be able to use the tool btrfs-corrupt-block, which is used by the btrfs developers to test the filesystem.

Now, like I said, I haven't had a lot of time to play around with btrfs, so I don't know the exact usage of this tool. But with it, you should be able to corrupt an offline filesystem, which will be fixed when the corrupted file is read (assuming that you have set up RAID or something so that there's another copy to use).

share|improve this answer
2  
Awesome find! Assuming that btrfs-corrupt-block is actually written to be a genuine test and not a "trick" by the btrfs developers, this should fit the bill exactly. –  allquixotic Jan 16 at 17:46
    
@allquixotic if you want to learn more about btrfs, there's a great talk from linux.conf.au 2012. like I said, btrfs-corrupt-block is used by the developers, so it wouldn't be very useful if it was a trick :) –  strugee Jan 16 at 17:51
3  
@allquixotic That's the beauty of open source: you can look at the source code of btrfs and check it! Sure, this wont be an easy task, but if you really want to, you can do that. –  Bakuriu Jan 16 at 19:10
    
@Bakuriu I'm completely aware of that. I never seriously doubted that btrfs-corrupt-block isn't a sincere test, since it would be found out very quickly by someone poking at the source, and used as negative PR against Oracle (at least; as well as any other btrfs developers/contributors). It was just an off-the-cuff comment. –  allquixotic Jan 16 at 21:08
    
I wonder if the OP ( @Oli ) want's to corrupt a block (ie, the filesystem structure) or a file (ie, the content of a file??)... And I believe the btrfs claim of self-healing is about the former, not the latter? [how would a filesystem know which bit was flipped in a file? some kind of CRC?]. This answer is probably in the right, though, so +1. [but it may change more than a "single bit" ? or change something that can be healed more easily than a random bit happening "anywhere" ?] –  Olivier Dulac Jan 17 at 8:52
  1. Get the value of a single sector on the block device (e.g. /dev/sda1) with an offset of 1 million sectors offset (just an example):

    sudo dd if=/dev/sda1 of=/root/mysector bs=512 count=1 skip=1M
    

    This arbitrary chosen 1M * 512 bytes offset is just to make sure you're out of the metadata part of the filesystem and actually on a sector which contains data.

  2. Edit the raw sector data by changing the content with a hex editor. See for example Need a good hex editor for Linux.

  3. Put back the sector on the drive with the if and of arguments reversed:

    sudo dd if=/root/mysector of=/dev/sda1 bs=512 count=1 seek=1M
    
share|improve this answer
1  
This won't help him test unless the 1-millionth block is actually a part of a file. How can he look up what block a particular file starts on? –  Darth Android Jan 16 at 16:46
3  
That's so almost there. If you can lock the dd command onto the exact position of the file, this is probably the best way to approach this. –  Oli Jan 16 at 16:47
    
@Oli Yes, I know how to do that for the Ext-family of filesystems, but I don't have that much experience with btrfs. Let me see if I can find a way. –  gertvdijk Jan 16 at 16:52
1  
@Oli : you can simply have a loop, dd outpouting block by block (ie, like the above, but "skip=N", N being in 1..max) until you can grep a line from the file you wish to edit [try to generate a line that won't happen anywhere else... for example take it from a password generator, and long enough?]. Then you edit that particular block. remount, test if the change was reverted (which I doubt, see my comment in the top answer... there seems to be a confusion between file data (=content) and the filesystem structure itself (= how files and their content are organized) ?) –  Olivier Dulac Jan 17 at 8:55

@Oli - hi, I'm Jim Salter, the guy who actually wrote that article. I was working with a virtual machine, which made things simpler. What I did is started out with a JPEG file, and opened it up in a hex editor. The particular one that I used was Bless, which you can install in Ubuntu with a simple apt-get install bless.

After opening up the JPEG in Bless, I hit page down a few times to get well into the "meat" of the JPEG, and then just highlighted about fifty bytes worth of data, and copied and pasted it into a text editor (in my case, gEdit). This gave me something to search for.

Now I saved the JPEG into each array on the VM. The storage behind the arrays were a series of .qcow2 files. Once I'd saved the JPEG into the arrays, I could load the .qcow2 files associated with each array into Bless, and search them - they weren't very large, being nothing but the JPEG and some metadata - for that fifty byte pattern I'd highlighted and copied out of the JPEG. Voila, I had the block to corrupt! At this point, I could just manually edit individual bytes of the JPEG as stored on the VM's virtual disk using Bless - and, importantly, do so in exactly the same way on each array.

The only wrinkle is that in the case of the RAID5 array tested in the article, I had to make sure I edited the actual copy of the data in the stripe, and not the parity for the stripe itself - it was a small image on an otherwise empty array, so there wasn't any data in the FOLLOWING block in the stripe, making the parity block contain the data unaltered from the data block. If I'd accidentally edited the parity block instead of the data block, the image would have shown up as unchanged.

One final note - you don't NEED virtual machines to do this - you could do the same things in the same ways with bare metal; it would just be more of a pain in the butt because you'd need to work with entire raw drives instead of with nice small .qcow2 files, and you'd either have to pull the drives and put them in a different machine, or boot into a live (or just alternate) environment to mess with them. (I tested ZFS's data healing in exactly this way, but on real bare metal machines, 7-ish years ago when I first got interested in next-gen filesystems.)

Hope this helps!

share|improve this answer

You could try a small program that will perform FIBMAP ioctl(2) on the opened file.

By quick web search, I found this blog post http://smackerelofopinion.blogspot.tw/2009/06/fibmap-ioctl-file-system-block-number.html detailing how to do this - it will even give you a link to a sample program that you can compile and run yourself.

$ git clone git://kernel.ubuntu.com/cking/debug-code
$ cd debug-code/block-mapper-fibmap
$ make
$ sudo ./fibmap /path/to/your/image-file.jpg

This is exactly the way hdparm --fibmap (mentioned by @falconer) is implemented.

After finding the block numbers you could employ dd gongfu to modify the file, like @gertvdijk sketched out. Or maybe you could just modify the fibmap.c program above to do the bit flip for you, directly writing to the device file bypassing the file system layer (three parameters for the program: 1. the path to the file, 2. device file containing the file system, 3. offset and bit you want to modify).

(Disclaimer: I have not tested and cannot guarantee that the FIBMAP ioctl(2) will work for a file in loopback device or btrfs file system, but I would strongly expect it does. I am guessing hdparm will check the device type before performing the ioctl(2) on the file and is hence failing.)

share|improve this answer
sudo hdparm --fibmap /PATH/TO/FILE

will give you the LBAs where the file is located. After this you can use @gertvdijk 's answer.

share|improve this answer
    
Unfortunately this doesn't seem to work. It's spitting out a 0,39: device not found in /dev It's either because it's btrfs or (more likely) because I'm using it on loopback-held files. I'll try and get this done with a "proper" VM. –  Oli Jan 16 at 18:17
    
@Oli Hmm. I thought that hdparm works on every file system but maybe it is not so. –  falconer Jan 16 at 18:22

Your Answer

 
discard

By posting your answer, you agree to the privacy policy and terms of service.

Not the answer you're looking for? Browse other questions tagged or ask your own question.