[nycbug-talk] eurobsdcon

[Brown, James Jim]

> -----Original Message-----
> From: Pete Wright [mailto:pete at nomadlogic.org]
> Sent: Monday, May 24, 2004 9:17 AM
> To: Brown, James (Jim)
> Cc: NYC Bug List
> Subject: Re: [nycbug-talk] eurobsdcon
> 
> 
> Brown, James (Jim) wrote:
> 
> >
> >  
> >
> >>-----Original Message-----
> >>From: G.Rosamond [mailto:george at sddi.net]
> >>Sent: Sunday, May 23, 2004 10:31 PM
> >>To: Jan Schaumann
> >>Cc: NYC Bug List
> >>Subject: Re: [nycbug-talk] eurobsdcon
> >>
> >>
> >>
> >>On May 23, 2004, at 10:25 PM, Jan Schaumann wrote:
> >>
> >>    
> >>
> >>>Dru <dlavigne6 at sympatico.ca> wrote:
> >>>
> >>>      
> >>>
> >>>>-contributing documentation, tutorials, reviews
> >>>>        
> >>>>
> >>>This is one point that can't be stressed enough.  Documentation is
> >>>something that anybody can contribute to - even though 
> >>>      
> >>>
> >>getting really
> >>    
> >>
> >>><snip>
> >>>      
> >>>
> >
> >Documentation is important, but there is a more critical need, IMHO.
> >
> >Open Source software needs reliable testing frameworks.  Most Open
> >Source projects of substance are large (> 50K lines of code) and
> >most do not have a reliable, rigorous testing framework. 
> >My perception is that Open Source testing consists mainly of just
> >getting as many people as possible to run the project code
> >and submit bug reports.  While helpful, this is spotty testing.
> >
> >
> >The Perl Test Harness is a step in the right direction.  
> >Test programs are written for specific features of a module
> >and compared against expected output.  These can be run by
> >anyone, on any hw/sw platform that runs Perl.
> >
> >The BIND 9 test suite is another good example, although in this
> >case, it's not easy to set up and run.
> >
> >
> >Consider challenging the Open Source community to develop testing
> >frameworks for every project.  'make test' should perform a 
> >complete suite of tests for the project that can be reused
> >for every release.
> >
> >  
> >
> yea i totally agree with you there.  and hopefully OSDL has 
> started to 
> take on this role, and maybe other organizations will follow. 
>  i do not 
> even know of any opensource software testing out there, is there any?
> 


I don't know of any 'one-size-fits-all' software testing suite.  
I do know of projects that have formed their own test suites
and make them available- some right in the project Makefile
as I noted above.  But not every project does this.  'make test'
is not universally adopted.

The point I'm trying to emphasize here is that spotty testing
must go away.  Open Source needs rigorous testing if it is going
to seriously compete with commercial products.

Of course, the specific tests would vary by project.  A web proect
needs network based web testing, while a graphics project needs
a lot of graphics.  What's common is that they both have large 
code bases that can be modularly (is that a word?) tested.
The modularized frameworks would cover the standard 'unit, 
integration, system' tests in their own way.

Perhaps a branding scheme- 
'Project Foo conforms to Open Source Test Framework v. 1.0'
with a spiffy logo would work.  Personally, I'd rather see
somethat like that instead of 'Powered by Foo'.


Best Regards,
Jim B.

PS- I'm not a testing engineer, but I have done a fair
amount of software development over the years.  I've developed
my own tests for my own code, and in a few cases, I've had to 
endure the rigors submitting it to a larger enterprise test
environment.  It was a helpful experience, though somewhat 
painful.



Note:  The information contained in this message may be privileged and
confidential and protected from disclosure.  If the reader of this message
is not the intended recipient, or an employee or agent responsible for
delivering this message to the intended recipient, you are hereby notified
that any dissemination, distribution or copying of this communication is
strictly prohibited.  If you have received this communication in error,
please notify us immediately by replying to the message and deleting it from
your computer. Thank you.  ThruPoint, Inc. 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://lists.nycbug.org/pipermail/talk/attachments/20040524/1a1276de/attachment.html