[nycbug-talk] Approach for the NFS cluster

[Pete Wright]

On 19-May-09, at 12:42 PM, Jerry B. Altzman wrote:

> on 5/19/2009 3:30 PM Isaac Levy said the following:
>> Wordup All,
>> Keeping data state is always a trick.
>>> Worth the pursuit.
>> Indeed.... down the rabbit hole though...,
>
> I'd be interested to find *anyone* who's gotten a good roll-your-own  
> HA
> NFS / HA NAS setup working on BSD. I've seen a lot of DRBD/Heartbeat
> setups in "semi-production" mode, but that was all on Linux. I know I
> live in a somewhat cloistered, rarefied world, but ...


the trick with a true HA setup for NFS is keeping your file handles in  
sync b/w N nodes in your cluster.  this is a pretty serious issue -  
sure you can replicate your data b/w N nodes using tricks like DRBD,  
which frankly has a pretty negative performance penalty, or using a  
global filesystem behind your NFS servers (hadoop, gpfs, etc).

I wouldn't consider these solutions to be highly available - highly  
replicated maybe, but I'm not sure they increase your "availability"  
in failure scenarios.  this is why NetApp, EMC, Isilon and Sun get to  
charge you lots of money.  Its a hard problem to do correctly and  
reliably.  Although when it works it is nice to see that you can be  
reading from a file, failover your NFS server, and not be disrupted :)

FWIW - the new Sun 7000 "open storage" platform has the ability to be  
configured in an HA setup (check out the 7410 which i've just finished  
installing actually).  This platform is built on top of OpenSolaris  
and ZFS.  Perhaps there is some doc around opensolaris.org that  
describes how they handle failover.  i know it's not a BSD solution -  
but perhaps there are some bits they use that can be ported over to BSD.

oh yea...both NetApp and Isilon are built on FreeBSD too btw!  So I  
guess I have seen this setup on BSD before :)

-pete