Package java.io

Class StreamTokenizer

The StreamTokenizer class takes an input stream and parse it into tokens, allowing the tokens to be read one at a time. The parsing process is controlled by a table and a number of flags that can be set to various states. The stream tokenizer can recognize identifiers, numbers, quoted strings, and various comment styles.

Valid input values for the stream tokenizer are between '\u0000' and '\u00FF'. The input values are regarded as characters with five possible attributes: whitespace, alphabetic, numeric, string quote, and comment character. Each character can have zero or more of these attributes.

Instances of the string tokenizer can have the following flags:

• Line terminators are to be returned as tokens or treated as whitespace that merely separates tokens.
• C-style comments are to be recognized and skipped.
• C++-style comments are to be recognized and skipped.
• Characters of identifiers are converted to lowercase.

Public Constructors