Josef “Jeff” Sipek

Totally awesome day! I submitted Unionfs to the usual places (linux-kernel, fsdevel, and the key people), then I stayed up all night. In the morning, I got a form for permission to enroll in the graduate version of compilers (I’d much prefer lex & yacc to some made up java thing the undergrad course uses). At around 10, I decided to head home and get some sleep. I woke up about 8 hours later, and checked my email. I replied to a lot of comments/questions by Andrew Morton and some other people, and when I finally managed to check the rest of the inbox, I saw:

Jan 08 akpm@osdl.org   ( 236) + unionfs-documentation.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 107) + lookup_one_len_nd-lookup_one_len-with-nameidata-argument.patch added to -mm
Jan 08 akpm@osdl.org   ( 138) + unionfs-branch-management-functionality.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 649) + unionfs-common-file-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 733) + unionfs-copyup-functionality.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 299) + unionfs-dentry-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 313) + unionfs-file-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 319) + unionfs-directory-file-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 326) + unionfs-directory-manipulation-helper-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 995) + unionfs-inode-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 572) + unionfs-lookup-helper-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 743) + unionfs-main-module-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 344) + unionfs-readdir-state.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 501) + unionfs-rename.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 263) + unionfs-privileged-operations-workqueue.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 168) + unionfs-handling-of-stale-inodes.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 228) + unionfs-miscellaneous-helper-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 402) + unionfs-superblock-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 233) + unionfs-helper-macros-inlines.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 552) + unionfs-internal-include-file.patch added to -mm tree
Jan 08 akpm@osdl.org   (  87) + unionfs-include-file.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 218) + unionfs-unlink.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 109) + unionfs-kconfig-and-makefile.patch added to -mm tree

Unionfs is now in -mm!

If you actually look at the next -mm changelog, you only see one patch containing all of Unionfs, as Andrew decided to use the git tree that I set up (gitweb) on kernel.org.

So, I moved my Unionfs development from a bunch of i386 boxes, to 2 x86_64 servers. They are sweet :) I still have to get used to looking in arch/x86_64/ and include/asm-x86_64/ instead of their i386 equivalents which I have used for years.

So, it would seem that Linus has gone insane. His announcement of 2.6.18 is rather odd — I didn’t expect him to go for the Internation Talk-like a Pirate day thing…

Ahoy!

She’s good to go, hoist anchor!

Here’s some real booty for all you land-lubbers.

There’s not too many changes, with t’bulk of the patch bein’ defconfig updates, but the shortlog at the aft of this here email describes the details if you care, you scurvy dogs.

Header cleanups, various one-liners, and random other fixes.

Linus "but you can call me Cap’n"

Oh, and I got a simple patch into the kernel. I have about 8 more of similar ones that should get into 2.6.19-rc1. Oh, and of course, I should resubmit Unionfs in the next week or two. Grr…so much to do!

Hrm, the ever-ending TODO list currently contains two papers I want to read. Since the TODO list doesn’t actually exist anywhere beyond my head, I figured that I’d post the links to the papers here, and maybe someone who reads this will bug me, in turn making me actually read them.

• Soft Updates: A Technique for Eliminating Most Synchronous Writes in the Fast Filesystem
• Stackable Design of File Systems (I already read his SOSP’95 paper about the topic.)

So, here’s a mini-rant…There are just as many XFS complaints as ext3 complaints on the linux-kernel mailing list. Yep. It is that simple. I can’t stand the fact that some people make a big deal out of complaints about XFS, but are oddly silent (or ignorant?) of the fact that there are just as many “problems” with Ext2/3. I’m not even considering Ext4, as it is in development.

So, finally, after long time of trying to get Unionfs into the Linux kernel, we submited it to linux-kernel, fsdevel, cc’ing all the people that should be cc’d (Al Viro, Christoph Hellwig, and Andrew Morton.)

So, I was in #linuxfs on OFTC, and a whole discussion happened about XFS and ext3. Eric Sandeen was working on fixing up few bugs in ext3 to make it work on 16TB of storage. I couldn’t help but mention XFS. The discussion then evolved into XFS is a large pile of code that is really nasty in places (I do agree with that) but it still performs very well. Eric pasted a link to this image (I copied it for archival purposes.) which shows the code size of XFS over time. It is actually kind of scary.

You can look at this post as either a sequal to a post I made few days ago, or as another sequal to a post I made about a year ago.

So, while doing some more cleanup, Dave Quigley discovered this gem in Unionfs:

for (bindex = bstart - 1; bindex >= 0; bindex--) {
        err = copyup_file(dentry->d_parent->d_inode,
                          file, bstart, bindex,
                          file->f_dentry->d_inode->i_size);

        if (!err)
                break;
        else
                continue;
}

The particular piece of interest is the if statement:

if (!err)
        break;
else
        continue;

What does it mean? If there is no error, we break out of the loop. Makes sense. If there is an error, we go to the top of the loop. Makes sense…right? Well, what happens if you reach the end of a for loop? We go to the top of the loop! See the brain damage? We are at the end of the loop, so we don’t need an explicit continue statement.

Here’s the if statement the way it should be (for completeness):

if (!err)
        break;

Oh, so simple, it almost brings a tear into one’s eye.

Think of this as a sequel to my post about a year ago.

The following peice of code is from Unionfs’s copyup_permission(). It is (or should be) a simple function that is supposed to copy the permissions from one inode to another. Well, I can’t help but remember this one slide from Greg Kroah-Hartman’s keynote this year at Ottawa Linux Symposium. Here’s the slide:

The quote definitely applies to copyup_permission(). It seems very clear that it “evolved” to something very odd. Anyway, I shall torment you no longer, here is the code:

static int copyup_permissions(struct super_block *sb,
                              struct dentry *old_hidden_dentry,
                              struct dentry *new_hidden_dentry)
{
        struct iattr newattrs;
        int err;

        print_entry_location();

        newattrs.ia_atime = old_hidden_dentry->d_inode->i_atime;
        newattrs.ia_mtime = old_hidden_dentry->d_inode->i_mtime;
        newattrs.ia_ctime = old_hidden_dentry->d_inode->i_ctime;
        newattrs.ia_valid = ATTR_CTIME | ATTR_ATIME | ATTR_MTIME |
                        ATTR_ATIME_SET | ATTR_MTIME_SET;
        /* original mode of old file */
        newattrs.ia_mode = old_hidden_dentry->d_inode->i_mode;
        newattrs.ia_gid = old_hidden_dentry->d_inode->i_gid;
        newattrs.ia_uid = old_hidden_dentry->d_inode->i_uid;
        newattrs.ia_valid |= ATTR_FORCE | ATTR_GID | ATTR_UID | ATTR_MODE;
        if (newattrs.ia_valid & ATTR_MODE) {
                newattrs.ia_mode = (newattrs.ia_mode & S_IALLUGO) |                
                        (old_hidden_dentry->d_inode->i_mode & ~S_IALLUGO);
        }

        err = notify_change(new_hidden_dentry, &newattrs);

        print_exit_status(err);
        return err;
}

It was actually Seth Arnold that noticed that the condition will ALWAYS be true because ATTR_MODE is set in the line just above it. Furthermore, if one eliminates the if statement and replaces newattr.ia_mode in the assignment with what it is set to just few lines before (right after the comment) he gets:

newattrs.ia_mode = (old_hidden_dentry->d_inode->i_mode & S_IALLUGO) |
                   (old_hidden_dentry->d_inode->i_mode & ~S_IALLUGO);

If you are up to speed with bitwise operations, you’ll realize that it can be simplified to:

newattrs.ia_mode = old_hidden_dentry->d_inode->i_mode;

Brilliant!

Friday was kind of interesting. The talks were little weaker, but there were some interesting ones. For example, Matt Mackall’s Towards a Better SCM: Revlog and Mercurial talk was a nice way to learn how Mercurial stores the history. I got tired so I went back to the hotel, and fell asleep for few hours. I woke up just in time to head over to the conference center for the 20:00 Stackable file systems BOF. That was interesting. A lot of useful people showed up, including Christoph Hellwig, Ted Tso, Val Henson, Steve French (Samba/CIFS guy), Jan Blunck,Eric Van Hensbergen (plan9 fs implementation in Linux) and many more. Topics included limited stackspace (4K on i386), cache consistency, locking, and nameidata structure brain damage.

As we planed before, after the BOF we invited everyone over to the hotel for some snacks and drinks. That’s where things got really interesting. I spent a lot of time with Jan Blunck and Eric Van Hensbergen talking about the proper way to do unions. Three people, three different ways to union files :)

After that we had some fun with the stack space issue and Reiserfs (and Hans’s approach to open source).

stack space

There should be a competition "who can create the largest storage stack without overflowing the stack." For example, unionfs on top of gzipfs on top of versionfs on top of device mapper on top of md on top of scsi over ethernet on top of ppp tunneled thought ssh …. you get the idea

Reiserfs

Apparently, Christoph once fixed something trivial in reiserfs code, he sent the patch to Hans, and in return he got a gigantic legal document (I know exactly what he is talking about as I have submitted a patch once as well). Well, he didn’t like it, so he gave Hans a bunch of options one of which included certain sum of money for copyright to the code. Interestingly enough, Hans accepted. Too bad I didn’t know about this story before, I might have made some money of off my one line fix to the reiserfs section in the Kconfig file. :)

I have to say, I really want to look at how plan9 works thanks to the conversation I had with Eric Van Hensbergen. It seems to have a lot of really good ideas. Hrm, now that I think about it, I might install it one of my old boxes. Yeah, I know I am odd.