Too Busy For Words - the PaulWay Blog

Dogma seems to pervade computing just as much as in religion. Two particular pieces of dogma that I've watched through the years are "Linux file systems don't fragment" and "XFS can't be shrunk". When anyone declares things like this out of hand, and I can see obvious counterexamples of other systems that work similar that don't obey these dictums, I start asking why.

On the first, I had a brief chat with Ted T'so at LCA 2006 in which I ascertained that, yes, ext2 and most other Unix filesystems will fragment files if there is not enough space to put the file in one contiguous block. This only happens, of course, when the filesystem is nearly full, and is exacerbated by older filesystems (e.g. FAT) being fairly dumb about where to put new files in the first place - if you just keep on moving through the disk looking for a free block of appropriate size then you're going to wrap around sooner or later and start splitting useful blocks into non-useful pieces.

This is only half of the problem with file system fragmentation, by the way. The other half is the idea of concentrating access for similar things (e.g. files in the same directory) in a locale on the disk to minimise the seek times. As far as my understanding goes, this is done better in Unix than in DOS (and, to a certain extent, Windows) because a directory will be given an extent of the disk, so as you fill up the /lib directory all its files are close to eachother on disk. This doesn't address the problem that you rarely read the entire /lib directory and then move on, nor the fact that most disks have faster data transfer on the outside of the disk (where track zero resides), so distributing files further into the disk than they need to be does result in more slowdown, as well as increasing the average seek times. I've yet to see anything good on this so one of my projects is to research this topic and give a talk at LCA sometime on it.

The other dogma about being unable to shrink an XFS filesystem is interesting. My research so far is that, in September 2005, a guy named Iustin Pop submitted a patch that would allow shrinking an XFS filesystem while it was online, including both the kernel routines to shrink the filesystem if the 'disappearing' area was completely unallocated and the necessary userspace programs to actually move the data around to achieve this. This seems to be the result of a lot of work on Iustin's part, after talking to Steve Lord in 2004 where Steve explained the problems of shrinking the filesystem. He notes:

The only way to really do this is a complete tree walk of all the metadata in the filesystem looking for any inode which has a metadata or data block in the allocation groups being freed, and reallocate that space elsewhere.

And? So? Is that so difficult? We walk through all the metadata all the time, when doing backups and stuff. Who said it had to be instantaneous? I've sent an email to Iustin to see what's become of his patch, because as far as I can see nothing has happened to this in the interim, and as late as June 2006 people (on the Red Hat mailing lists) were still spouting the dogma that shrinking XFS was completely impossible.

Beware of assumptions... they make an ass out of U and mptions. Or something. :-)