[nycbug-talk] dragonflybsd: process sharing/virtual kernels

[Pete Wright]

On Sun, Sep 03, 2006 at 11:02:39AM -0400, Bjorn Nelson wrote:
> All,
> 
> I was wondering what people think about being able to cluster at the  
> OS level.  Matthew Dillon is proposing virtual kernels with caching  
> as an easier alternative to their goal of process sharing between  
> machines:
> http://www.shiningsilence.com/dbsdlog/index.php/2006/09/02/1853.html
> 
> The end result will be that you can have two or more machines operate  
> as effectively one operating system.  Virtual kernels seem pretty  
> similar to the VM stuff that has been heating up lately with vmware  
> and xen.  The main difference is that to provide the clustering  
> ability, you need to add a component for data synchronization.  Could  
> this be built on top of the xen work being added to freebsd?   
> Possibly using gated and carp together to take care of delegating the  
> network/filesystem resources.
> 
> Is this basically vmotion from vmware?  Anyone have any experiences  
> in this area?
> 

this all seems pretty close the UML (user mode linux) which may or may
not be a good thing.  vmotion seems pretty interesting on the surface,
but there are still alot of concerns I personally have with it (cough
master station only run's on XP cough).  i'm actually doing some testing
with vmotion now...so hopefully i'll have better input in a month or
so...

> This looks like this is going to be a show stopping feature in the  
> future when choosing an operating system for general serving  
> purposes.  I can just imagine many of my fears of hardware redundancy  
> evaporating when we get to the point of having failover at the OS level.
> 
maybe, but you still will need redundant hardware regarless...think IBM
(or sun for that matter) mainframes...just because you can move your os
instance around does not mean you will just adandon the hardware right,
and i'd rather hotswap bad parts when possible than messing around
migrating production applications on the fly...

> The benefit of this is basically what Google has realized with their  
> cluster of cheap computers.  You don't have to worry about redundancy  
> at the host level nearly as much because a host is no longer a single  
> point of failure, and you don't have to worry about accurately  
> predicting the hardware required for your application as you can just  
> add another host to the pool if it's not fast enough.  Now, it's easy  
> to see this and say it but as with all issues it's rarely black and  
> white.  You may still want to mirror your OS drives, to lessen the  
> effect of the higher rate of failure of disks, and you may still want  
> to do some homework for purchasing hardware as at a certain point you  
> may have realized that you should have started with a faster base  
> system as a building block (decreasing returns due to increase in  
> overhead per performance of adding another machine, then again can  
> this negated by "weighting" the machines so that faster machines  
> serve more?).
>
exactly, we actually don't even bother mirroring disks...it's cheaper
(and quicker) to just swap out a whole unit in the case of a hardware
failure...or just rebuild the OS (via xcat www.xcat.org).  in cases like
this a VM would just hinder the performance of the
cluster/renderfarm/etc.  although i'd say HPC clusters or whatever fill
a different niche than where VM's are trying to address.

 
> What other implications are here?  Will sans be obsoleted?
> 
my take on the whole VM craze lately is this:
it's great if you have many app's that need custom or isolated execution
environments - *only* if said app's do not require alot of system
resources.  it seems to me that VM environment works great in a place
like say a bank that has fairly predictable usage patterns...well the
application that is going to total today's reciepts will need alot of
system resources at 4:30pm...and the one that does all wire transfers
will need alot of system resources at 2:00am.  so there is no reason why
these two applications can't share hardware.

but, if you have an unpredictable usage pattern (or are constantly using
<%90 of your system resources) then I frankly don't think you will gain
much from a VM.

will, san's be obsoleted?  heh...funny you mention that.  we are
actually doing RnD on SAN's for some VM stuff we are doing.  so yea, my
best guess is that SAN's will become more important as the use of VM's
grow's.

-p


-- 
~~oO00Oo~~
Peter Wright
pete at nomadlogic.org
www.nomadlogic.org/~pete
310.869.9459