[kaffe] Compare kaffe and IBMJava2-131 Java Virtual Machine.

[Greg Wooledge]

Dalibor Topic (robilad at yahoo.com) wrote:

> I assume that the Freenet developers lurking on this
> mailing have some experience with respect to
> scalability of kaffe on their platforms, as they use
> kaffe with 
> a) tons of threads
> b) to run crypto code
> c) lots of network activity

> Matthew, Greg, could you provide some
> anectodes/numbers?

Matthew is a Freenet developer; I'm a user, documentation writer, and
unofficial tech support helper.

I'm using Kaffe to run Freenet on OpenBSD because, as far as I know,
there is no other usable Java virtual machine for OpenBSD.  The
Linux x86 Java binaries from Sun will run, to a certain extent, but
they did not work for me when running Freenet.  (The Linux emulation
layer is obviously not 100% complete/bug-free.  The major symptom
was that reads from a socket that didn't already have data waiting
didn't "block" as they were supposed to; they returned without any
data.  Since Freenet is supposed to provide network services, this
made it unusable.)

I'm configuring Kaffe with "./configure --with-includes=/usr/local/include
--with-libraries=/usr/local/lib --with-engine=jit3".  I also have to
edit the libtool file by hand after the configuration, to change
"need_version" from "no" to "yes".  Apart from this, it compiles and
installs cleanly.

I have two problems with Freenet under Kaffe: crashing and memory
usage.  I've posted gdb stack traces of Kaffe core dumps to this
mailing list already.  Since the FD_ISSET assertion (jthread.c:1750)
was removed from CVS recently, I no longer get that particular one;
but I'm still getting this one occasionally:

assertion "!INTS_DISABLED()" failed: file "exception.c", line 398

However, I'm getting dramatically *fewer* crashes with CVS Kaffe
than I did when I started using Kaffe (1.0.7).  (It's also worth
mentioning that Kaffe 1.0.6 and earlier will not work with Freenet
at all.  The OpenBSD Kaffe port is still 1.0.6, and therefore less
than useful for me.)

In terms of memory usage, a long-running Freenet node creeps continually
upward.  I've got the ulimits set so that Freenet cannot exceed 256 MB
of memory, and I'm running the java command with "-mx 224M".  After
a few days, it will hit this limit (if it doesn't crash first).  It's
not supposed to do that; Linux users report that Freenet's memory usage
under Sun's Java VM remains steady at something like 30 MB (depending
on which build of Freenet is being used, which version of the Java VM,
etc.).

Apart from these two issues, Freenet runs quite well under Kaffe for me.

Performance is extremely difficult to measure with Freenet because
the amount of work being done by a live node is unpredictable.
Data requests from other nodes could occur at any time.

The single most intensive operation the node performs, at least from
the end user's perspective, is encoding/decoding large files with FEC
(Forward Error Correction).  Large files in Freenet are broken into
"segments" with a maximum size of 128 MB.  Each segment consists of
128 or fewer data blocks, and a number of check blocks (currently 50%
of the data blocks, so a maximum of 64).  Retrieving any 128 of these
192 blocks is sufficient to reconstitute the whole segment, but
decoding the 128 MB segment can take several minutes.  On my 1300 MHz
Duron, it's a bit over 2 minutes, wall-clock time, but this can be
deceptive because the node is not idle during this operation; it's
still processing other requests.  (It's also not entirely CPU-bound;
there is a considerable amount of disk activity associated with
this, writing the blocks to temporary files during the fetching,
reading them back in during the decoding, and writing the reconstituted
segments out to the final destination.)

I'm using the Java FEC library that's supplied with Freenet.  There
is also supposed to be a native x86 FEC library, but I have not
tried it.  I'm not even sure what the compatibility requirements are.

Since I can't run Freenet under Linux on this machine to give numbers
which would be useful for comparison purposes, all I can say is
that I'm pleased with Kaffe's performance under heavy loads.

> On a side note, is there a FreenetMark ?

There is (or was) a simulator which fired up several Freenet nodes
on a single machine in order to test the networking protocols,
routing, and so on.  I don't think it was designed to measure
performance; but rather, correctness and scalability of the Freenet
code and heuristics.

The Freenet developers might have more information about it; it was
mentioned recently on the Freenet development list.  (The thread
started here:
<http://hawk.freenetproject.org:8080/pipermail/devl/2003-February/004408.html>
and continued here:
<http://hawk.freenetproject.org:8080/pipermail/devl/2003-March/004415.html>
It seems that the mailing list archives can't handle threads that
span an end-of-month boundary.)

-- 
Greg Wooledge                  |   "Truth belongs to everybody."
greg at wooledge.org              |    - The Red Hot Chili Peppers
http://wooledge.org/~greg/     |
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 187 bytes
Desc: not available
Url : http://kaffe.org/pipermail/kaffe/attachments/20030306/17ace797/attachment-0003.pgp