Home Company Services Portfolio Contact us nav spacer

About resizing and recovering filesystems

by Izak Burger — posted on May 06, 2008 05:58 PM — last modified Apr 02, 2009 12:48 PM —

Murphy's law kicked in and we ended up with a file system that was 99% full, but with no files on it.

A colleague of mine did something horrible last night. He resized an LVM volume along with the file system on it without making a backup first. Murphy's law kicked in and we ended up with a file system that was 99% full, but with no files on it.

This morning, after enjoying the luxury of some five hours sleep, it dawned on me that the problem might well be the journal of said ext3 filesystem. Knowing that an ext3 filesystem is an ext2 file system with extra goodies, I tried mounting it as ext2:

# mount -t ext2 -o ro /dev/vg/volume /mnt/recover
mount: /dev/vg/volume already mounted or /mnt/recover busy

First line of defense: turn that error into a google search. This eventually tells me that the problem must lie with device mapper holding on to the device. After a bit of deliberation, I remove the mapped device:

dmsetup remove /dev/mapper/vg-volume

Then I do an lvscan to recreate it:

# lvscan
... snip ...
inactive '/dev/vg/volume' [70.00 GB] inherit
... snip ...

And activate it:

# lvchange -a y /dev/vg/volume

And just like that, I could mount it as an ext2 filesystem and get my files back.

The moral of this story, I think, is to remove the ext3 journal before you resize things, and add it back afterwards.

Document Actions