Josef “Jeff” Sipek

Totally awesome day! I submitted Unionfs to the usual places (linux-kernel, fsdevel, and the key people), then I stayed up all night. In the morning, I got a form for permission to enroll in the graduate version of compilers (I’d much prefer lex & yacc to some made up java thing the undergrad course uses). At around 10, I decided to head home and get some sleep. I woke up about 8 hours later, and checked my email. I replied to a lot of comments/questions by Andrew Morton and some other people, and when I finally managed to check the rest of the inbox, I saw:

Jan 08 akpm@osdl.org   ( 236) + unionfs-documentation.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 107) + lookup_one_len_nd-lookup_one_len-with-nameidata-argument.patch added to -mm
Jan 08 akpm@osdl.org   ( 138) + unionfs-branch-management-functionality.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 649) + unionfs-common-file-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 733) + unionfs-copyup-functionality.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 299) + unionfs-dentry-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 313) + unionfs-file-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 319) + unionfs-directory-file-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 326) + unionfs-directory-manipulation-helper-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 995) + unionfs-inode-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 572) + unionfs-lookup-helper-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 743) + unionfs-main-module-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 344) + unionfs-readdir-state.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 501) + unionfs-rename.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 263) + unionfs-privileged-operations-workqueue.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 168) + unionfs-handling-of-stale-inodes.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 228) + unionfs-miscellaneous-helper-functions.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 402) + unionfs-superblock-operations.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 233) + unionfs-helper-macros-inlines.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 552) + unionfs-internal-include-file.patch added to -mm tree
Jan 08 akpm@osdl.org   (  87) + unionfs-include-file.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 218) + unionfs-unlink.patch added to -mm tree
Jan 08 akpm@osdl.org   ( 109) + unionfs-kconfig-and-makefile.patch added to -mm tree

Unionfs is now in -mm!

If you actually look at the next -mm changelog, you only see one patch containing all of Unionfs, as Andrew decided to use the git tree that I set up (gitweb) on kernel.org.

Hrm, the ever-ending TODO list currently contains two papers I want to read. Since the TODO list doesn’t actually exist anywhere beyond my head, I figured that I’d post the links to the papers here, and maybe someone who reads this will bug me, in turn making me actually read them.

• Soft Updates: A Technique for Eliminating Most Synchronous Writes in the Fast Filesystem
• Stackable Design of File Systems (I already read his SOSP’95 paper about the topic.)

So, here’s a mini-rant…There are just as many XFS complaints as ext3 complaints on the linux-kernel mailing list. Yep. It is that simple. I can’t stand the fact that some people make a big deal out of complaints about XFS, but are oddly silent (or ignorant?) of the fact that there are just as many “problems” with Ext2/3. I’m not even considering Ext4, as it is in development.

So, finally, after long time of trying to get Unionfs into the Linux kernel, we submited it to linux-kernel, fsdevel, cc’ing all the people that should be cc’d (Al Viro, Christoph Hellwig, and Andrew Morton.)

So, I was in #linuxfs on OFTC, and a whole discussion happened about XFS and ext3. Eric Sandeen was working on fixing up few bugs in ext3 to make it work on 16TB of storage. I couldn’t help but mention XFS. The discussion then evolved into XFS is a large pile of code that is really nasty in places (I do agree with that) but it still performs very well. Eric pasted a link to this image (I copied it for archival purposes.) which shows the code size of XFS over time. It is actually kind of scary.

You can look at this post as either a sequal to a post I made few days ago, or as another sequal to a post I made about a year ago.

So, while doing some more cleanup, Dave Quigley discovered this gem in Unionfs:

for (bindex = bstart - 1; bindex >= 0; bindex--) {
        err = copyup_file(dentry->d_parent->d_inode,
                          file, bstart, bindex,
                          file->f_dentry->d_inode->i_size);

        if (!err)
                break;
        else
                continue;
}

The particular piece of interest is the if statement:

if (!err)
        break;
else
        continue;

What does it mean? If there is no error, we break out of the loop. Makes sense. If there is an error, we go to the top of the loop. Makes sense…right? Well, what happens if you reach the end of a for loop? We go to the top of the loop! See the brain damage? We are at the end of the loop, so we don’t need an explicit continue statement.

Here’s the if statement the way it should be (for completeness):

if (!err)
        break;

Oh, so simple, it almost brings a tear into one’s eye.

Think of this as a sequel to my post about a year ago.

The following peice of code is from Unionfs’s copyup_permission(). It is (or should be) a simple function that is supposed to copy the permissions from one inode to another. Well, I can’t help but remember this one slide from Greg Kroah-Hartman’s keynote this year at Ottawa Linux Symposium. Here’s the slide:

The quote definitely applies to copyup_permission(). It seems very clear that it “evolved” to something very odd. Anyway, I shall torment you no longer, here is the code:

static int copyup_permissions(struct super_block *sb,
                              struct dentry *old_hidden_dentry,
                              struct dentry *new_hidden_dentry)
{
        struct iattr newattrs;
        int err;

        print_entry_location();

        newattrs.ia_atime = old_hidden_dentry->d_inode->i_atime;
        newattrs.ia_mtime = old_hidden_dentry->d_inode->i_mtime;
        newattrs.ia_ctime = old_hidden_dentry->d_inode->i_ctime;
        newattrs.ia_valid = ATTR_CTIME | ATTR_ATIME | ATTR_MTIME |
                        ATTR_ATIME_SET | ATTR_MTIME_SET;
        /* original mode of old file */
        newattrs.ia_mode = old_hidden_dentry->d_inode->i_mode;
        newattrs.ia_gid = old_hidden_dentry->d_inode->i_gid;
        newattrs.ia_uid = old_hidden_dentry->d_inode->i_uid;
        newattrs.ia_valid |= ATTR_FORCE | ATTR_GID | ATTR_UID | ATTR_MODE;
        if (newattrs.ia_valid & ATTR_MODE) {
                newattrs.ia_mode = (newattrs.ia_mode & S_IALLUGO) |                
                        (old_hidden_dentry->d_inode->i_mode & ~S_IALLUGO);
        }

        err = notify_change(new_hidden_dentry, &newattrs);

        print_exit_status(err);
        return err;
}

It was actually Seth Arnold that noticed that the condition will ALWAYS be true because ATTR_MODE is set in the line just above it. Furthermore, if one eliminates the if statement and replaces newattr.ia_mode in the assignment with what it is set to just few lines before (right after the comment) he gets:

newattrs.ia_mode = (old_hidden_dentry->d_inode->i_mode & S_IALLUGO) |
                   (old_hidden_dentry->d_inode->i_mode & ~S_IALLUGO);

If you are up to speed with bitwise operations, you’ll realize that it can be simplified to:

newattrs.ia_mode = old_hidden_dentry->d_inode->i_mode;

Brilliant!

Friday was kind of interesting. The talks were little weaker, but there were some interesting ones. For example, Matt Mackall’s Towards a Better SCM: Revlog and Mercurial talk was a nice way to learn how Mercurial stores the history. I got tired so I went back to the hotel, and fell asleep for few hours. I woke up just in time to head over to the conference center for the 20:00 Stackable file systems BOF. That was interesting. A lot of useful people showed up, including Christoph Hellwig, Ted Tso, Val Henson, Steve French (Samba/CIFS guy), Jan Blunck,Eric Van Hensbergen (plan9 fs implementation in Linux) and many more. Topics included limited stackspace (4K on i386), cache consistency, locking, and nameidata structure brain damage.

As we planed before, after the BOF we invited everyone over to the hotel for some snacks and drinks. That’s where things got really interesting. I spent a lot of time with Jan Blunck and Eric Van Hensbergen talking about the proper way to do unions. Three people, three different ways to union files :)

After that we had some fun with the stack space issue and Reiserfs (and Hans’s approach to open source).

stack space

There should be a competition "who can create the largest storage stack without overflowing the stack." For example, unionfs on top of gzipfs on top of versionfs on top of device mapper on top of md on top of scsi over ethernet on top of ppp tunneled thought ssh …. you get the idea

Reiserfs

Apparently, Christoph once fixed something trivial in reiserfs code, he sent the patch to Hans, and in return he got a gigantic legal document (I know exactly what he is talking about as I have submitted a patch once as well). Well, he didn’t like it, so he gave Hans a bunch of options one of which included certain sum of money for copyright to the code. Interestingly enough, Hans accepted. Too bad I didn’t know about this story before, I might have made some money of off my one line fix to the reiserfs section in the Kconfig file. :)

I have to say, I really want to look at how plan9 works thanks to the conversation I had with Eric Van Hensbergen. It seems to have a lot of really good ideas. Hrm, now that I think about it, I might install it one of my old boxes. Yeah, I know I am odd.

I was going to post this on the 28th, but then I decided to go to bed, now that I found the draft lying around, I decided to amend it.

Original post…

Quota are awesome! I am just playing with them on a disposable installation of Debian Sarge and it is lot of fun!

# modprobe quota_v2
# mount / -o remount,quota
# touch /aquota.user
# touch /aquota.group
# quotacheck -am
# edquota username

And now what I think about quota…

• The user interface is very nice
• The developer interface just plainly SUCKS

Yes, the user-land utilities are great in linux, the just work! But the quotactl() call is horrendous…first of all it is not portable. Sure, it work if you don’t make drastic changes to the environment, like changing the kernel quota system (ver. 1 in 2.4.21 and earlier, ver. 2 in 2.4.22 and later, including the 2.5/2.6 kernels). Also if you use something as non-standard as XFS, you are out of luck…the quotactl syscall takes a different command when using XFS. It’s just aweful.