[nycbug-talk] BSD Cluster Filesystem Roundup

[Miles Nordin]

>>>>> "ak" == Andy Kosela <akosela at andykosela.com> writes:

    ak> I don't think open source BSD systems today has any enterprise
    ak> level production ready clustered filesystem.

I've the impression none of the clustering stuff is enterprizze level
production-ready.

At the unigroup talk before last:

  http://www.unigroup.org/unigroup-0901.html

the speaker's employer was making fat sacks working around banks'
obstinate insistance on things like ``we're a Dell shop'' or ``only
the built-in Ethernet chip is supported''.  They were so insistant on
substituting cargo-cult conservative choices for shopping and testing
that they're left out of the whole cluster market.  They are not
buying myrinet/infiniband/CEE switching fabrics, not using RDMA at
all, just wiring up their data centers like a floor full of accountant
cubes and desktops.  The only clustering thing they use is IP
multicast, and it soudned to me as if they didn't even understand how
to use it properly because the speaker was saying the filters in the
Ethernet MAC were responsible for blocking multicast traffic in which
the node isn't interested, while if you set it up properly by avoiding
the 224.0.0.0/24 block (which is never filtered by switches) and
making sure all destinations differ in the last 23 bits, and possibly
by making sure there's a 1 somehwere in bits 23 - 8 to avoid confusion
with the magical never-filtered /24, then the switches and routers
should do filtering for you before packets even hit the hash filter
inside the desktop's Ethernet chip.  I bet they were saying ``oh i
dunno billy bob i never used anything outside that 224.0.0.0/24 Class
C, seems like it'd just be asking for trouble, and on a Production
Network, too.  I mean we move a lot of money here so we can't afford
to be experimenting---we need to be using stuff that's READY and and I
think IP multicast must still be partway experimental technology in
the industry because I don't fully understand it myself so I don't
want to go fooling around with these IP blocks that aren't ready for
the enterprise yet.  Let's use Jim's IP scheme instead.  no, no I
don't want to do testing, we have a deadline.  Jim can we see the
powerpoint for that again?''  anyway that's a bit of a tangent, but
doesn't it seem like the more legitimate the clustering stuff is, the
less enterprisey productiony it is, the more rickety in-house toolkits
which are called tools and the fewer ``middleware packages'', the more
non-Dell hardware?

The little gold star for GlusterFS is that it's the only one of the
lot in which BSD has any prayer of participating.  I think it'd be a
better plan, if you wanted to actually build something based on
clustered FS, to dive in more broadly than deeply.  Set up three or
four of them.  You won't even know what quirks to look for in testing
until you've been exposed to a few.

also I like the way they let you store a POSIX filesystem inside a db4
file.  :)

What exactly was the cluster filesystem in Tru64 though?  I hope you
are not just talking about DCE/DFS?

I'm not sure VMS clusters will be a helpful guide because weren't they
meant to be only two or three nodes because the machines running VMS
were so big?  Such design would be completely different from the
hundreds-of-nodes clusters people are interested in now.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 304 bytes
Desc: not available
URL: <https://lists.nycbug.org:8443/pipermail/talk/attachments/20090224/2b7e043d/attachment.bin>