[kaffe] Release status...

[Jim Pick]

Hi,

I announced that I had made a branch for the 1.1.5 release almost a 
month ago, but I haven't made the final release yet.  :-(

So I thought I'd drop a note to the list on what's taking me so darned 
long to do it...

Part of it is due to the usual excuses (work, real life, etc.) and my 
own procrastinating.  But part of the reason is due to the fact that I 
want to make some improvements to the process.

There are several steps to making the release:

  a) Reviewing the changes and summarizing them for the release
     announcement.

  b) Building it in several places and actually trying to use it to
     run some software.  That's been the extent of my testing for
     the "development" releases.

  c) Fixing any obvious configure issues.  There are so many ways to
     configure Kaffe, I always seem to run into issues nobody else
     has.

  d) Fixing any "make dist" type problems that show up.  In the final
     steps of making the release.  Typically, I need to cycle through
     the build multiple times, as documented here:

http://www.kaffe.org/cgi-bin/viewcvs.cgi/kaffe-project-services/release-process/HOW-TO-MAKE-A-RELEASE?rev=1.1&content-type=text/vnd.viewcvs-markup

Naturally, this would be less work if I did this incrementally as
development progressed.  But I always end up doing this at the end.

With the previous releases, it took me a good 8-12 hours to do all
the steps.  That's just too long, and it's not fun work.

This time around, the amount of work involved has increased a bit, 
because of the longer interval since the previous release (partly due to 
my procrastinating).  Also, the fact that so much new good stuff has 
gone into it means that I encountered a lot of regressions all over the 
place, to the point I wasn't really happy with the final result.

(I think the code in CVS right now is quite a bit better than the state 
it was in when I made the 1.1.5 branch a month ago)

When I encounter Kaffe-related problems on one of my various machines 
around the house or the office, then I've got to make a decision.  I can 
either take some time to try to fix/debug/understand the problem 
(perhaps asking the list for help), or I can skip past the problem so I 
can get the release out sooner.  Naturally, I don't like skipping past
problems -- I'd rather fix them, but there are really are a 
near-infinite number of bugs to fix, so when it's late in the release 
cycle, it's usually better to skip past them and just get the release 
out.  There's a trade-off though -- if I skip past too many bugs, then 
all those bugs are going to go into the release, which isn't good.

Anyways, on to the process improvement part of the tale:

Ultimately, I want to automate the whole release process.  When checkins 
get made, they should be automatically tested for regressions on a whole 
variety of different platforms and in different configurations.  The 
code should only be considered release-worthy when it passes all the 
testing we throw at it.  The release process should be as simple as 
selecting a particular build that passed all the testing.  This way, we 
can certify that a particular build works and was tested on all the 
various platforms and configurations we support.

The distinction between a "production" release (which is one of my goals
for this year) and a "development" release should primarily be that the
production release will have passed a lot more testing.

If we had such a system in place, then the testing and fixing grunt work 
that gets left until release time (and lumped onto me) would instead be 
done incrementally as development occurs, and shared by everyone.

I actually started coding up a custom regression testing and reporting 
system last year, and I probably got it 50% done.  I think I could 
probably get it partially deployed with about a week's worth of work.

Does anybody have any objections to me holding off on the 1.1.5 release 
until I get the regression testing system deployed on the server?

Initially, I'll just set up some testing environments on some of the 
servers I have access too.  After I get something working, I'll try to 
document as much as possible so other people can help out with the testing.

If I could get the regression testing system set up to do most of the 
release testing, that would really make the release process simply 
(almost automatic).  It would be a lot more fun too.

Sorry for keeping everybody in suspense for so long...

Cheers,

  - Jim