StreamTokenizer not recognising "-"

[masher]

Hello all

I'm having a some trouble using StreamTokenizer to split up an input stream into tokens, and then put them into an ArrayList.

My problem is that when the ST encounters a "-", ttype is set to 45. As a consequence, "-"'s are not passed into my ArrayList.

I know that the return of 45 is the ASCII value of "-". The ST API specs state that 0-9, "." and "-" are treated as numerals.

I have tried with and without wordChar settings. With and without parseNumbers()


Sample input/output:

-69 -> -69.0 (ttype = -2)
69 -> 69.0 (ttype = -2)
69- -> 69.0 + ttype = 45
69-- -> 69.0 + ttype = 45 + ttype = 45
i- -> i- (ttype = -3)
i-- -> i-- (ttype = -3)
i-j -> i-j (ttype = -3)
i - j -> i +ttype=45 + j


Question 1: Why is the ttype = 45 and not -2 (TT_NUMBER)?



Question 2: How can I fix it?


Thanks in advance


Matthew


This is my code:

      // This reads in the file
      BufferedReader inFile = new BufferedReader (new
                                 FileReader ("Date.java"));

      // Creates the stream for the file
      StreamTokenizer streamT = new StreamTokenizer(inFile);


      streamT.wordChars(33,47); //characters make up words .
      streamT.wordChars(58,64); // 48-57 are numerals 58-64 are some punctuation stuff
      streamT.wordChars(91,96); // 65-90 are already wordchars 91-96 are more punctuation
      streamT.wordChars(123,126);// 97-121 are already wordchars 123-126 are more punctuation
      streamT.parseNumbers(); // this says to recognise numbers as numbers.
      streamT.eolIsSignificant(true); // the end of line is significant, and returns a different token, not just whitespace


      //  creates array to hold the seperate tokens from the program
      ArrayList programTokens = new ArrayList();


      int printIndex = 0; // this is here for debugging

      // while the next token isn't the end of file..
      while(streamT.nextToken() != StreamTokenizer.TT_EOF)
      {

         System.out.println(streamT.ttype); // this is here for debugging
         
         // if the type of variable is a word, then add the word
         if(streamT.ttype == StreamTokenizer.TT_WORD)
         {
            programTokens.add(streamT.sval);
         }

         // if the type of variable is a number, then add the number
         else if(streamT.ttype == StreamTokenizer.TT_NUMBER)
         {
            // ArrayList stores objects, so I convert my number to a string
            String stringInput = Double.toString(streamT.nval);
            programTokens.add(stringInput);
         }

         // if the type of variable is EOL, then print EOL.
         else if(streamT.ttype == StreamTokenizer.TT_EOL)
         {
            programTokens.add("ENDOFLINE");
         }


        System.out.println(programTokens.get(printIndex)); // this is here for debugging
         printIndex++; // this is here for debugging

      }