StreamTokenizer

StreamTokenizer

java.io

---

Java 1.0

This class performs lexical analysis of a specified input stream and breaks the input into tokens. It can be extremely useful when writing simple parsers. nextToken( ) returns the next token in the stream; this is either one of the constants defined by the class (which represent end-of-file, end-of-line, a parsed floating-point number, and a parsed word) or a character value. pushBack( ) pushes the token back onto the stream, so that it is returned by the next call to nextToken( ) . The public variables sval and nval contain the string and numeric values (if applicable) of the most recently read token. They are applicable when the returned token is TT_WORD or TT_NUMBER . lineno( ) returns the current line number.

The remaining methods allow you to specify how tokens are recognized. wordChars( ) specifies a range of characters that should be treated as parts of words. whitespaceChars( ) specifies a range of characters that serve to delimit tokens. ordinaryChars( ) and ordinaryChar( ) specify characters that are never part of tokens and should be returned as-is. resetSyntax( ) makes all characters ordinary. eolIsSignificant( ) specifies whether end-of-line is significant. If so, the TT_EOL constant is returned for end-of-lines; otherwise , they are treated as whitespace. commentChar( ) specifies a character that begins a comment that lasts until the end of the line. No characters in the comment are returned. slashStarComments( ) and slashSlashComments( ) specify whether the StreamTokenizer should recognize C- and C++-style comments. If so, no part of the comment is returned as a token. quoteChar( ) specifies a character used to delimit strings. When a string token is parsed, the quote character is returned as the token value, and the body of the string is stored in the sval variable. lowerCaseMode( ) specifies whether TT_WORD tokens should be converted to all lowercase characters before being stored in sval . parseNumbers( ) specifies that the StreamTokenizer should recognize and return double-precision floating-point number tokens.

 public class  StreamTokenizer  {  // Public Constructors   #  public  StreamTokenizer  (InputStream  is  );  1.1  public  StreamTokenizer  (Reader  r  );  // Public Constants  public static final int  TT_EOF  ;  =-1  public static final int  TT_EOL  ;  =10  public static final int  TT_NUMBER  ;  =-2  public static final int  TT_WORD  ;  =-3   // Public Instance Methods  public void  commentChar  (int  ch  );        public void  eolIsSignificant  (boolean  flag  );        public int  lineno  ( );        public void  lowerCaseMode  (boolean  fl  );        public int  nextToken  ( ) throws IOException;        public void  ordinaryChar  (int  ch  );        public void  ordinaryChars  (int  low  , int  hi  );        public void  parseNumbers  ( );        public void  pushBack  ( );        public void  quoteChar  (int  ch  );        public void  resetSyntax  ( );        public void  slashSlashComments  (boolean  flag  );        public void  slashStarComments  (boolean  flag  );        public void  whitespaceChars  (int  low  , int  hi  );        public void  wordChars  (int  low  , int  hi  );  // Public Methods Overriding Object  public String  toString  ( );  // Public Instance Fields  public double  nval  ;        public String  sval  ;        public int  ttype  ;   }