[RFC] Automated QA build system

[Alex Perez]

Fred Kiefer wrote:
> Adam Fedor wrote:
>
> > On Feb 23, 2005, at 1:55 PM, Alex Perez wrote:
> >
> > > Two for Two. This is a touchy subject, but it just keeps happening 
> > > over and over and over again. Richard, would it be possible for you 
> > > to actually /test/ the code you commit to CVS? (not the code you 
> > > write on your machine, the actual code which actually gets comitted 
> > > to CVS, the code other people must use)? The entire GNUstep community 
> > > will benefit from higher QC, Lets learn from our mistakes, instead of 
> > > repeating them bi-weekly.
> >
> > Why are you getting so upset over two missing headers? It's quite a 
> > simple mistake, particularly for a patch that would probably have 
> > never gotten in if Richard hadn't spent some time on an OS he never uses.
> >
> > Please be more patient.
>
> I want to second Adam on this, especially as it was me who noticed the 
> missing files in the first place. CVS is not the greatest of all version 
> management tools, it is very easy to miss out on submitting single 
> files. (You would know, if you were coding)

Is this supposed to be an insult? If so, it fell flat on its face. I've 
used CVS for years. Also, the GNUstep website itself is in CVS, which I 
update periodically on a fairly regular basis. Surely Richard has far 
more experience with CVS than I do.

> All of us really should try hard to avoid this from happening, but when 
> it happens its fine to just correct it as soon as possible.

If you tested the actual code which gets committed (which IMHO is really 
a basline requirement) then this would not happen. Things which break 
the entire build cycle should be taken more seriously.

Anyways, my comments were merely meant as constructive criticism. It's 
frustrating to check out CVS two times in a week and have it fail to 
build for such a trivial reason.

After a discussion on #GNUstep with Stefan Urbanek (stiivi), I will pass 
along his idea/suggestion. Some sort of automated QA mechanism would be 
great, where, upon a new CVS check-in to -base, for example, it would be 
checked out by an automated QA system and compiled on Linux/x86 as a 
baseline test. The results of the compilation would be placed on a 
webpage. If compilation breakage to any part of -core happened, a 
message would be sent to -dev notifying the core team that this had 
happpened, the offending file, and the version of the file in CVS at 
which the breakage occurred.

Does anyone think this would be valuable? It could be expanded 
upon/combined to do unit testing as well as pretty much any other 
Quality Assurance task you could dream up.

I'd appreciate hearing everyone's thoughts on this (that includes all of 
the core devs).

I'd also like to set this up for win32, since that seems to be the thing 
that breaks fairly often as other patches go into -core.

Cheers,
Alex Perez