[kaffe] Re: Regression/build testing using Xenofarm

[Dalibor Topic]

Hi Jim,

Jim Pick wrote:
> On Mon, 28 Jul 2003 14:22:21 +0200
> Dalibor Topic <robilad at kaffe.org> wrote:

> I did some of the ground work on it this weekend.
> 
> The good news is that JSTL works great on Jetty (providing I drop in
> some of the Apache XML stuff into the BOOTCLASSPATH to override Kaffe's
> XML stuff).  Actually compiling the JSPs using kjc is a tad bit slow and
> a memory hog, but it all works.  :-)

Cool, another interesting application/toolkit that 'works on kaffe'.

> I've got Friday off - so I'm shooting for installing the first version
> of it on Saturday.  :-)

I'm looking forward to it. I've just fixed kaffe's make dist after my 
patch from yesterday, and I'm trying to see if I can get the alpha-linux 
jitter to work again.

>>I'm not quite sure about 2) though. What should go into the reporting 
>>format? A lot of build tools just output (not very formatted) text to 
>>stdout and stderr. Or do you have filters in mind, that XMLize the 
>>output of 'make', for example?
> 
> 
> The XML "reports" would mostly just identify the build number, the
> platform, the test performed, and whether or not it was "in progress",
> "pass" or "fail".  It would also have additional metadata that might
> provide an http link to the compile log.

that sounds cool.

> Developers would drop these .xml files into a designated directory
> underneath their home directories on pogo.kaffe.org.  Eventually, I'd
> like to write up a way to set up web accounts and http uploading or
> WebDAV on kaffe.org so that non-developers could also easily submit
> reports.

Can't you just add another layer of indirection so that non-developers 
could publish the results on their servers, and an ant task could 
collect them automatically?

> Every few minutes, a cron job will run an ant task which reads all the
> .xml files (maybe using XOM), and dumps them, or maybe just the
> new/changes ones, into a SQL database (hsqldb?). The web output will be
> a set of JSPs that just pulls data out of the SQL database. I thought of
> using XSLT to do the processing, but that doesn't scale as nicely as
> SQL.

Argh, too many acronyms exception ;)

hsqldb is nice, I've played with it a couple of months ago. It should 
run fine on kaffe. How about XOM, does that work with kaffe?

> The initial web output is going to look something like tinderbox, except
> it will be heirarchical, as we have literally hundreds of testable
> combinations of tests/platforms/configurations.

the curse of being very configurable ;)

> There's not actually that much code involved.
> 
> We need a way of assigning build numbers to the various checkins.  I
> was thinking of just counting the Changelog entries to come up with
> a number.  It shouldn't be too hard to write scripts that take the
> output of any type of test (compilation, make check, mauve, SpecJVM,
> Gump, stability tests, etc.), make an XML file, and upload the
> results.

Counting changelog entries is fine if everyone uses the ChageLog to 
document changes. That seems to work quite well so far. Other than that 
you could use the unix date as a long, for example. I doubt kaffe's CVS 
will be still around in 2037 ;)

cheers,
dalibor topic