Fun with Opterons, SATA, and INNODB

原文来自： http://www.feedblog.org/2007/05/fun_with_optero.html

We've been doing a lot of performance analysis of MySQL and SATA disks over the last few days and I wanted to get some thoughts out in the open to see what you guys thought.
Now that Debian Etch is out we're seriously looking at making the jump to a full 64bit OS with 8G of memory on each of our boxes.
This also involves benchmarking SATA and potentially migrating away from MyISAM which, while very trusty, is seriously showing its age for us.
First off is raw IO tuning.
XFS benchmarks show about a 20% performance boost from bonnie++ over ext3. This is pretty obvious. Tuning readahead with 'blockdev' didn't seem to help performance much. SATA seems pretty tuned when compared to hdparm and IDE drives.
After fully tuning it seems we can get about 90MBps out or our disks. Not to shabby. The Linux md driver on RAID 0 didn't seem to boost IO performance much (and I'm pretty disappointed). Even with a 1M chunk size (we also tested 64k) we were only seeing about 110MBps. Each drive can (in parallel) handle about 85MBps concurrently so it's not an issue with the controller. It might be an issue where the RAID really starts to show performance in concurrent reads.
We might just end up using MySQL 5.1 with partitioning on top of INNODB which should allow us to utilize both disks.
XFS still scares me though. Apparently (according to Andrew Morton) there's a long standing bug with XFS where it writes the metadata log first and then the data second. This can lead to a race condition where the box crashes before the data is written and has an inconsistent metadata log. Your file will then become corrupt with a series of 0000s in the middle. Ouch.
One issue which will become hard to deal with is the fact that Etch out of the box doesn't support NCQ. I couldn't get the drives to enable NCQ on bootup and apparently this is a known issue with libata. I'm sure this problem will settle down in the next two months as Etch matures a bit. Hopefully I won't have to patch the kernel.
INNODB vs MyISAM
We're about 50-70% write bound so having fast writes is very important for us. INNDBO seems to be the clear winner here. With the support for a write ahead log as well as other operations it basically destroyed MyISAM.
We even cheated and created a 1G RAM disk and created a MyISAM table there. MyISAM was still about 30% slower than INNODB since it ended up pending 100% of the CPU (userspace) building the table.
We also benchmarked Postgres vs INNODB and while the results are very inconclusive it's looking like INNODB will beat Postgres here as well (especially for our workload).
More thoughts as they come. I can't wait to deploy this stuff.