[nycbug-talk] Measuring Disk Performance
spork at bway.net
Thu Dec 15 14:55:13 EST 2011
On Dec 15, 2011, at 9:13 AM, Isaac Levy wrote:
> On Dec 12, 2011, at 7:31 PM, Pete Wright wrote:
>> On Mon, Dec 12, 2011 at 11:57:41AM -0500, Isaac Levy wrote:
>>> Hi All,
>>> Disks: I need to measure disk performance across heterogeneous UNIX boxes, and I'm sick of bonnie++ for a bunch of reasons I won't' waste time on here.
>>> I know from conversations that a few folks here have their own ways of testing disks-
>>> I'd really like to know what people do to measure general disk performance? e.g. really simple tests:
>>> - r/w/d large files
>>> - r/w/d small files
>>> - disk performance when directories contain large numbers of files
>>> I commonly have need to test things like:
>>> - different block sizes
>>> - different inode allocations (UFS/ext3
>>> - different filesystem partition layouts
>>> - different filesystem features (think ZFS fun)
>>> Any thoughts, experiences, urls or shell utils to share?
>> hey ike!
>> there is actually a pretty decent chapter on measuring disk and
>> filesystem performance in "High Performance PostgreSQL 9.0".
> Is it this book:
> "PostgreSQL 9.0 High Performance"
> Author: Gregory Smith
> Publisher: Packt Publishing
It is. I just got that, haven't read the whole thing yet, but skimmed it and it is chock-full of good stuff. I'm just starting to come up with a benchmark methodology for getting hard numbers comparing a spinning disk postgres box vs. and all-SSD postgres box. That book (and the author, and lots of insanely great people on the postgresql-performance list) has steered me away from some boring and time-wasting benchmarks.
There are also a bunch of tools for postgres that help you test with your own data/workload:
But yeah, even pgbench is going to give your disks a good workout.
I can't emphasize enough just how many hours you can lose if you start browsing the performance mailing list archives. Fascinating stuff in there.
>> they talk
>> about using a tool, bundled with bonie++, called zcav that will track
>> transfer rates from the begining to end of a disk subsystem. it also
>> will output data into gnuplot friendly format for pcitures. i used this
>> quite extensivly while tuning a linux dataware house a while back.
>> other tools that I'm happy with are iozone and fio:
>> i find that when doing benchmarking of systems for eval purposes or
>> benchmarking i end up using a mixture of many different tools. i find
>> that differnent tools will stress different parts of a given i/o
>> subsystem. so i'd generally do something like:
>> - initial test using dd with variable blocksizes (dependent upon
>> underlying filesystem block size)
>> - several bonie++ tests, followed by some tests using iozone and fio
>> - depending on how system will be used in production i try some
>> application level tests. for a db - pgbench, webserver apache bench
> Understood, really comprehensive, and valuable!
> Have you posted your methodology and/or resulting tests anywhere?
> Conversely, what I'm looking to really do here, is more of a cave-man sanity test: test a few small common cases in a relative vaccum, ideally using UNIX builtins spitting a few integers back to stdout. (mix a bit of xargs with rm, mkdir, dd, etc…)
> I'm actually looking to avoid going down this path alltogether, as I'd rather focus this level of detail on measuring disk/system performance as the team hammers our apps.
>> I have also done some interesting testing using
> *That* looks rad- way off direct disk testing, but that's a very compelling toolkit...
> Thanks Pete!
> talk mailing list
> talk at lists.nycbug.org
Bway.net - New York's Best Internet www.bway.net
spork at bway.net - 212.982.9800
More information about the talk