SPECjvm98

[Jukka Santala]

On Sat, 23 Mar 2002, Dalibor Topic wrote:
> Do you have any specific benchmarks in mind? 

Not particularily, but despite the subject-line on this thread, I agree
with the view we should probably stay with open-source benchmarks. Ashes
looks like a good alternative, and even has Kaffe regression tests
section, oddly enough.

I think more interesting question is if we should try to agree on a
standard runtime environment; the compiler and libraries can have much
bigger effect on performance than the JVM/KaffeVM in question. Primarily,
this doesn't matter as long as the environment stays the same from test to
test, or changes are explictly noted, but to improve comparability of
optimizations between platforms etc. there might still be use for agreeing
on such. It seems like GCC 2.96 would be the preference, as this is common
standard, altough moving to 3.x series should be considered. How about
libraries, though? This is a tougher nut to crack, altough admittedly a
proper VM benchmark shouldn't depend so much on library performance.

 -Jukka Santala