[kaffe] Idea for a security compatible JIT

[John Richard Moser]

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

(I'm not subscribed, and I won't help you past this message, but it'd be
nice to know you're working on this :)

Please respond, even if just to say you read the message.  :)  I'm
interested in knowing whether or not you're interested in exploring this
idea at all; although this is all the information I can offer (I can't
help you rewrite your JVM or anything).

I've been looking at JVM and JIT for a while, and earlier today realized
that the problems I've encountered are completely solvable so that a
JIT, in the presence of flaws which cause it to produce poor code that
can be exploited, can run under a secured system which imposes
restrictions making current JIT methods impossible.  Such a system would
mitigate many of such security flaws.

A JIT compiler takes the Java byte code and produces native instruction
code, to my understanding.  This instruction code is generated in
memory, and then executed.  The problem with this is that certain
security systems (PaX) will not allow memory to transcend from
non-executable to executable, or to be writable and executable at the
same time.  These systems must remove the protections on the JIT engine.

There is a very simple way around this.  Outlined in the PaX
documentation is a set of possible ways to exploit a PaX protected
system.  Quoted below is a section of the documentation.

.  It is important to realize that our focus is not on the finding and
.  fixing such bugs but rather on prevention and containment of exploit
.  techniques.  For our purposes these techniques can affect the
.  attacked task at three different levels:
.
.    (1) introduce/execute arbitrary code
.    (2) execute existing code out of original program order
.    (3) execute existing code in original program order with arbitrary
.        data
.
[...]
.    - attack methods (2) and (3) are possible with 100% reliability if
.      the attacker needs advance knowledge of addresses and can derive
.      them by reading the attacked task's address space (i.e., the
.      target has an information leaking bug).
.
.    - attack methods (2) and (3) are possible with a small probability
.      if the attacker needs advance knowledge of addresses but cannot
.      derive them without resorting to guessing or a brute force search
.      ('small' can be further quantified, see the ASLR documentation).
.
.    - attack method (1) is possible if the attacker can have the
.      attacked task create, write to and mmap a file. This in turn
.      requires attack method (2), so the analysis of that applies here
.      as well (note that although not part of PaX per se, it is
.      recommended among others, that production systems use an access
.      control system that would prevent this venue of attack).

On a security note, on production systems using such an ACL system, the
permission to create, write, and mmap() such a file could be restricted
to the Java VM; thus, the security of the entire system in no case must
be weakened to facilitate the JVM, although the security pertaining to
the JVM may be weakened.

The JIT compiler is fully capable of creating, writing, and mmap()ing a
file.  It would be feasible for the JIT to create fully PIC output
(Position Independent Code), write it to a file, mmap() the file with
the PROT_READ|PROT_EXEC protections, and execute the code.  This would
allow it to run with full PaX restrictions.  This would in turn bring
benefits such as allowing PaX to catch an exploit in a buggy version of
the JIT compiler.

The hierarchy should be based on /tmp with certain permissions, as
below.  Obviously this would be different for non-unix targets; although
the idea of spitting out a DLL on Windows shouldn't be abandoned,
because it would help MS to converge to a level of security similar to
that which Linux is capable of now.

/tmp/.USER-jvm/
This directory holds the code generated for USER.  It should have the
permissions 0700, or drwx------.

/tmp/.USER-jvm/PID/
This directory holds the code generated by the JVM with the given PID.
It sholud also have the permissions 0700 or drwx------.

/tmp/.USER-jvm/PID/java.class.so
This file holds the code generated by the JIT for the class file
java.class.  It should be created with permissions 0600 (-rw-------) and
then have the permissions 0500 (-r-x------) set before mmap()ing it in.

The easiest venue for creating code that actually works is probably
going to involve actually generating a valid shared object and opening
it with dlopen().  This would give it a GOT for symbols and for
variables in the code.  The dlsym() function could be used to get the
address of the contained functions.

I'm not going to tell you this will be programmatically easy; even if I
believed that, I have never coded something like this, and am not
qualified to say such things.  I WILL tell you that it's possible; you
may possibly want to look into it.

I've been trying to propagate information about security technologies,
via web sites, articles, just talking to people, however I possibly can.
~ The first mention of this type of JIT solution came from me; I made
note of the possibility on http://d-sbd.alioth.debian.org/www/?page=pax as
part of an explanation about compatibility with PaX.

- --
All content of all messages exchanged herein are left in the
Public Domain, unless otherwise explicitly stated.

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.6 (GNU/Linux)
Comment: Using GnuPG with Thunderbird - http://enigmail.mozdev.org

iD8DBQFBYNyihDd4aOud5P8RAmPrAJ9SLgZA0arJtFk6gGGZ5+lN9bSTEACfbyrH
2s5kqZzMDEYk8zp6jDdNihY=
=Xxp/
-----END PGP SIGNATURE-----