[kaffe] race during thread creation and gc invocation

[Alexander Boettcher]

Hi,

i have a race condition in thread.c found which (may) causes Kaffe to
hang - for our L4 system it happens. In the function "startThread" in
kaffe/kaffevm/thread.c

startThread(Hjava_lang_VMThread* tid)
{ ...
        nativeTid = createThread(tid, &firstStartThread,
                                 jthread_current(),
                                 threadStackSize, &info);
  ...
        ksemGet(&THREAD_DATA()->sem, (jlong)0);
        linkNativeAndJavaThread (nativeTid, tid);
        ksemPut(&jthread_get_data(nativeTid)->sem);
}

a thread is created and later the native thread and the java object
are linked.
 
In the function "createThread" the pointer(to jthread_current()) is
saved - during the call to jthread_create - in (somewhat
jthread_t)->data.jlThread. In "linkNativeAndJavaThread" the pointer(
to tid) is saved in (somewhat jthread_t)->data.jlThread. This means,
different types of pointers are used in data.jlThread ... jthread* and
Hjava_lang_VMThread* .
 
The problem appears if between "createThread" and
"linkNativeAndJavaThread" the GarbageCollector is invoked (which
happens in my testcase sporatic). The gc calls for each thread -
during its work - "liveThreadWalker" in
kaffe/kaffevm/kaffe-gc/gc-refs.c, which assumes that jlThread is of
type Hjava_lang_VMThread. But the pointer between "createThread" and
"linkNativeThread" is of type jthread_t. In my case, Kaffe hangs and
does not return from KGC_markObject.

 The new created thread is already listed (createThread) in the
activeThread list of the current thread implementation (i looked at
unix-pthreads & unix-jthreads and it is also so for our l4thread
implementation). Therefore, the gc calls "jthread_walkLiveThreads" and
the function found the new created - but unready( still wrong
jthread_t pointer) - thread.
 

liveThreadWalker(jthread_t tid, void *private)
{
  Collector *c = (Collector *)private;
  threadData *thread_data = jthread_get_data(tid);
  Hjava_lang_VMThread *thread = (Hjava_lang_VMThread
*)thread_data->jlThread;

  printf("livethread jtid=%p unhandthread=%p
vmthread=%p\n",tid,unhand(thread)->thread,thread);
  KGC_markObject(c, NULL, unhand(thread)->thread);
  KGC_markObject(c, NULL, thread);
  ...
}

 
 So, jlThread can not be used in this form, a other pointer would be
necessary in the threadData structure or the gc may not assume that
jlThread is of type VMThread and has to handle it.
 
 What do you think ?
 
 Thanks - sorry for my english,
 
 Alexander Boettcher.