I hope to set up a better test system using only SCSI disks (which will properly honour write ordering) and run additional tests with xfs and jfs. It's been widely reported that xfs has a painfully slow unlink time, which is why it wasn't considered as part of the initial tests.
As can be seen from the following graph, the fastest filesystem under trivial load is ext2, but ext3 rapidly begins to outperform it under heavier load. ReiserFS is outperformed by all other tested filesystems and does not change under load. I am uncertain what causes the large drop in ext2 performance under load, but it was fairly consistent on all the runs. Journalled data mode outperforms ordered data mode on ext3 because it reduces the number of seeks necessary (by writing all the data out in a linear strip) without increasing the amount of data written (because the files would almost always be removed before they would get written back).
The following graph shows qmail's average concurrency while running the tests. Average concurrency is a measure of how hard qmail is working. Lower numbers at the same or better throughput are better.
All results are the geometric mean of four runs of 1000 seconds each.
The raw numbers produced by the scripts can be downloaded here. The "loop-1000" directory are the final results, the rest are test or discarded results. The scripts used to produce and analyze the data, as well as the Makefile used to produce the graphs can be found here.