[nycbug-talk] Lab environment

[Sunny Dubey]

On Thursday 20 May 2004 04:00 pm, George wrote:

> Conclusions would be something like, NFS runs amazingly faster than
> SMB, etc on NetBSD 1.5.3 with an Apple iBook g4 as client.

I think the a single BSD server will outlast the ibook, heh

>
> Very simple, but we can at least draw some useful conclusions.
>
> Maybe it's too simple. . .maybe we use three boxes. . .Net, Free, Open,
> and test multiple protocols on each box.  But then the tweaks and
> configs may change too much that it would distort the conclusions.  But
> we should decide on the client side box.

s#box#boxes

with different implementations and such.  The more heterogeneous clients are 
the more "real world" the simulated load will be.  ie:  Sambe<->Samba is 
supposed to be much faster than Samba<->Windows, so clearly there is 
something more going on.

I can write a super generic bourne script for fbsd/nbsd/obsd and linux.  Not 
too sure about os-x and I doubt windows can be scripted the way we want.


> But that angle has to be dealt with too. . .what tweaking and how much?
>   Every single open source related run in a test lab ends with some
> party or another complaining (usually legitimately) about custom
> configs.
>
> Maybe the best way to deal with this is to have an email to each
> project, stating our test parameters, and request their recommended
> settings, so that no complaining can happen later on. . .

I think the best way to go about optimizing would be to think like PHB 
would.  IE: use only the optimizations *officially documented* by the 
vendor.  Additionally these optimizations can only help the stability pool 
and must never decrease it.  This way if something were to break, we could 
go back to the official documention of whatever software and find easily 
whatever it was that caused our issues.

I don't think optimizations are not cheating in anyway because in any real 
world usage you wanna be able to get the most out of your money.  
Additionally quite often various defaults are left to safer-than-safe 
levels because the software installed has be generic enough to be able to 
be run on millions of random configurations.

Sunny Dubey

PS:  can this list be setup to default on reply to list.  This way we can 
all just hit reply and keep chatting.  I find CC'ing the person and the 
list to be annoying.