[nycbug-talk] shared disks for heavy I/O

Marco Scoffier marco at metm.org
Sun May 3 16:54:14 EDT 2009


Hello all,

I am looking for recommendations for a shared disk solution in the 3 to 
5TB range that can support heavy reading and writing.  Standalone ? 
Dedicated server ? Fiber channel ?  Budget is around $4000.  Has someone 
looked into this recently?  How does Gig-ethernet performance compare to 
other solutions in real world situations?

I am working with large data sets (presently 100G but eventually to 
become larger). I have to store intermediate steps of the 
computation to disk.  Often to speed things up I run parallel 
computations which each read and write 1 to 10G of data at the beginning 
and the end of the computation.  I have 2 16cpu servers with SATA disks 
which each exports its' disks to the other using NFS.  Very often all 
cpus on both systems are maxed at 100%.  Lately when ramping up the 
computations NFS has been locking up (even when only reading remotely 
and writing locally).  I/O has been slow (ls takes forever to return for 
eg.).  I think that we are probably asking too much of the current setup 
where we are both running computations and exporting NFS from the same 
machines.

Thanks,

Marco







More information about the talk mailing list