[kaffe] Bug report (java.io.StreamTokenizer)

[Ito Kazumitsu]

>>>>> ":" == Dalibor Topic <robilad@yahoo.com> writes:

:> A quick (i.e. not exhaustive) test would be to take a fresh tokenizer, reset it
:> so that all characters are ordinary, then pick a single character, and give it
:> different attribute combinations to see what precedence exists between them.

I wrote a simple program like this:

import java.io.*;
public class StreamTokenizerTest2 {
  public static void main(String[] args) throws Exception {
    StreamTokenizer tok = new StreamTokenizer(System.in);
    tok.resetSyntax();
    int c = args[0].charAt(0);
    for (int i=1; i<args.length; i++) {
       if (args[i].equals("S")) tok.whitespaceChars(c, c);
       if (args[i].equals("C")) tok.commentChar(c);
       if (args[i].equals("Q")) tok.quoteChar(c);
       if (args[i].equals("N")) tok.parseNumbers();
       if (args[i].equals("W")) tok.wordChars(c, c);
    }
    while (true) {
      int t = tok.nextToken();
      if (t == StreamTokenizer.TT_NUMBER) {
	  System.out.println(tok.nval + ": " + t);
      }
      else {
	  System.out.println(tok.sval + ": " + t);
      }
      if (t == StreamTokenizer.TT_EOF) break;
    }
  }
}

Running echo "121"| java StreamTokenizerTest2 1 N S and so on,
I found the precedence is not so simple in Sun's JDK.
It depends on in what sequence initialization methods are called.

bash$ echo "121"| java StreamTokenizerTest2 1 C S
null: 50
null: 10
null: -1
bash$ echo "121"| java StreamTokenizerTest2 1 S C
null: 10
null: -1
bash$