Kiev & out-of-memory

[Godmar Back]

> 
> I've downloaded beta3. Kiev works now! At least,
> it can compile itself (but generates wrong code - there should be
> some bugs in java libraries - I'll check this out).
> (Godmar, looks like you got old kiev008_g.jar - there is no
> the error you've reported with my distrib. I remember, I did some
> enhanchments in kiev.stdlib.Debug class, and they, may be,
> reflected only in new files - there are special bug-fixed jars
> on download page).
> 

 So, the verifier assertion violation is then gone for good in your case?

> Now, about $subj.
> 
> :-(
> 
> Results of running "kaffe kiev.Main ..." :
> total 1196617ms, max memory used = 56320 Kb
> 
> Results of running "java kiev.Main ..." (with TYA 1.0 jit) :
> total 634607ms, max memory used = 7918 Kb

If I may ask, where do you get these numbers from?
Also, how does Sun's JDK perform in this test?

> 
> I.e. 56 Mb memory usage to compare with 8 Mb :-(
> The compilation time difference may be coused by the memory, thus
> significantly more work for GC.

Please configure kaffe with --with-timing and see how much time is spent
marking the heap.

Also, when you run "kaffe -verbosegc", it should print a line for each gc.
It's be interesting to see the results of that.

Finally, configure kaffe with '--with-gcblock=mmap'; this should show
a further reduction in gc time.

> 
> There may be two reasons for such enormouse memory requirements:
> 1) Kaffe uses unoptimized allocation strategy
> 2) Kaffe does not properly collect the garbage.
> 
> First, Sun's java Object itself allocates not more then 12 bytes (plus some
> bits - for GC/syncronization, possibly). I did this test sometimes. Also,
> I know that those 3 words are a) pointer in the handler table to actual
> data,
> b) pointer in the handler table to virtual table of methods, c) header
> of object's data (I think - pointer to class structure of this object).
> Shure, pointer to VT is not required, but, looks like it greatly speedups
> method calls, and Sun decided to spend this 4 bytes ;-)
> So, can you explain the memory usage of kaffe objects? If it does not
> differ a lot (eight times difference for my compiler!) - the the problem
> in GC.

If I recall correctly, Kaffe currently incurs a 14 bytes per object
overhead.  8 bytes for pointers in a linked list, 4 bytes for the header
that points to the dtable, and 2x1 bytes for gc flags.

> 
> About GC. As I did explain in previous messages, kiev is two-pass
> compiler. Memory carefully freed after each file parsing. And memory
> structure is (huge) AST tree - so, any error in GC near the root oof tree
> may prevent collecting the memory :-( So, this is the most possible
> way to get those 56Mb... i.e., maybe, sometimes, memory cannot be
> collected... Well, I don't know. Please, how can we check this out?
> 

Well, the accounting in Kaffe's gc (what you see when you do a -verbosegc)
is probably somewhat broken at this point.

There is a lot of other reasons why memory use might be different.
For instance, not nulling out dead references may cause garbage to 
be collected later than desired.  Consider this code:

	g() {
		Tree	x = new LargeTree()
		f(x);
		continue without using x.
	}

	f(Tree x) {
		use x somehow;
		do something else without using x.
	}

In this example, x is dead after the call to f.  In fact, it could be
garbage collected right after f is done with it.  That's something Kaffe
currently doesn't realize.

Of course, there is also the potential that we don't pick up garbage.
A reproducible test case would be best to debug that possibility further.

	- Godmar