Kiev & out-of-memory

[Maxim Kizub]

>>
>> I've downloaded beta3. Kiev works now! At least,
>> it can compile itself (but generates wrong code - there should be
>> some bugs in java libraries - I'll check this out).
>> (Godmar, looks like you got old kiev008_g.jar - there is no
>> the error you've reported with my distrib. I remember, I did some
>> enhanchments in kiev.stdlib.Debug class, and they, may be,
>> reflected only in new files - there are special bug-fixed jars
>> on download page).
>>
>
> So, the verifier assertion violation is then gone for good in your case?

Yes. :-)
>> Now, about $subj.
>>
>> :-(
>>
>> Results of running "kaffe kiev.Main ..." :
>> total 1196617ms, max memory used = 56320 Kb
>>
>> Results of running "java kiev.Main ..." (with TYA 1.0 jit) :
>> total 634607ms, max memory used = 7918 Kb
>
>If I may ask, where do you get these numbers from?
>Also, how does Sun's JDK perform in this test?
It's not a test, actually, it's compiling the whole source tree of kiev
compiler itself.
As you may noticed, there is -gc N switch in the compiler. After (and
before) each file parsing I check the memory by call to java.lang.Runtime
totalMemory() & freeMemory() methods, and call System.gc() if need.
Of course, I _carefully_ null out all references to temporary data
(AST tree, memory used to generate bytecode, etc.) before these
calls. As side-effect I have a knowelege about max/current memory usage.
Time is measured by System.currentTimeMillis().
Sun's jdk reports java memory only, without needs of JVM itself.
In reality, memory usage is not 8Mb, but about 10Mb.
>>
>> I.e. 56 Mb memory usage to compare with 8 Mb :-(
>> The compilation time difference may be coused by the memory, thus
>> significantly more work for GC.
>
>Please configure kaffe with --with-timing and see how much time is spent
>marking the heap.
>
>Also, when you run "kaffe -verbosegc", it should print a line for each gc.
>It's be interesting to see the results of that.
>
>Finally, configure kaffe with '--with-gcblock=mmap'; this should show
>a further reduction in gc time.

I did these tests. --with-timing gives this results (as I remember):
total ~ 1100.
gc ~360.
(Sorry, looks like kaffe redirects these results not to stdout/stderr - I
saw
them on screen, but in log file "|tee err.log 2>&1" they not appear.
The same is for -verbosegc switch - results are not in log :-(
Visually, it starts from 10Mb and grows up to 56Mb (with jumps
down of about 10 to 20 Mb after each GC).
--with-gcblock=mmap does nothing. Slight increase of memory
usage about 0.5 Mb.

>>
>> There may be two reasons for such enormouse memory requirements:
>> 1) Kaffe uses unoptimized allocation strategy
>> 2) Kaffe does not properly collect the garbage.
>>
>> First, Sun's java Object itself allocates not more then 12 bytes (plus
some
>> bits - for GC/syncronization, possibly). I did this test sometimes. Also,
>> I know that those 3 words are a) pointer in the handler table to actual
>> data,
>> b) pointer in the handler table to virtual table of methods, c) header
>> of object's data (I think - pointer to class structure of this object).
>> Shure, pointer to VT is not required, but, looks like it greatly speedups
>> method calls, and Sun decided to spend this 4 bytes ;-)
>> So, can you explain the memory usage of kaffe objects? If it does not
>> differ a lot (eight times difference for my compiler!) - the the problem
>> in GC.
>
>If I recall correctly, Kaffe currently incurs a 14 bytes per object
>overhead.  8 bytes for pointers in a linked list, 4 bytes for the header
>that points to the dtable, and 2x1 bytes for gc flags.
So, this is not the reason.
>>
>> About GC. As I did explain in previous messages, kiev is two-pass
>> compiler. Memory carefully freed after each file parsing. And memory
>> structure is (huge) AST tree - so, any error in GC near the root oof tree
>> may prevent collecting the memory :-( So, this is the most possible
>> way to get those 56Mb... i.e., maybe, sometimes, memory cannot be
>> collected... Well, I don't know. Please, how can we check this out?
>>
>
>Well, the accounting in Kaffe's gc (what you see when you do a -verbosegc)
>is probably somewhat broken at this point.
>
>There is a lot of other reasons why memory use might be different.
>For instance, not nulling out dead references may cause garbage to
>be collected later than desired.  Consider this code:

No, Godmar, no. I do _carefully_ null all unneeded data. Without
this the compiler will waste gigabytes (mostly by JavaCC parser).

Also, I've runned my compiler with "-debug profile" switch, that
allows me count uncollected (my, not system) objects at the end
of compilation. Both kaffe & jdk have plenty of them, and kaffe has them
about
50%  to 100% percents more (I can't count exactly - too many data ;-) ).
But anyway, this still can't be the reason.

PS I saw a patch about inflate - will try it now and let's see results...

Regards
  Maxim Kizub