public interface IScanner
ToolFactory
.
The scanner is responsible for tokenizing a given source, providing information about
the nature of the token read, its positions and source equivalent.
When the scanner has finished tokenizing, it answers an EOF token (
ITerminalSymbols#TokenNameEOF
.
When encountering lexical errors, an InvalidInputException
is thrown.
This interface is not intended to be implemented by clients.
ToolFactory
,
Provisional API: This class/interface is part of an interim API that is still under development and expected to
change significantly before reaching stability. It is being made available at this early stage to solicit feedback
from pioneering adopters on the understanding that any code that uses this API will almost certainly be broken
(repeatedly) as the API evolves.
Modifier and Type | Method and Description |
---|---|
int |
getCurrentTokenEndPosition()
Answers the ending position of the current token inside the original source.
|
char[] |
getCurrentTokenSource()
Answers the current identifier source, after unicode escape sequences have
been translated into unicode characters.
|
int |
getCurrentTokenStartPosition()
Answers the starting position of the current token inside the original source.
|
int |
getLineEnd(int lineNumber)
Answers the ending position of a given line number.
|
int[] |
getLineEnds()
Answers an array of the ending positions of the lines encountered so far.
|
int |
getLineNumber(int charPosition)
Answers a 1-based line number using the lines which have been encountered so far.
|
int |
getLineStart(int lineNumber)
Answers the starting position of a given line number.
|
int |
getNextToken()
Read the next token in the source, and answers its ID as specified by
ITerminalSymbols . |
char[] |
getRawTokenSource()
Answers the current identifier source, before unicode escape sequences have
been translated into unicode characters.
|
char[] |
getSource()
Answers the original source being processed (not a copy of it).
|
void |
resetTo(int startPosition,
int endPosition)
Reposition the scanner on some portion of the original source.
|
void |
setSource(char[] source)
Set the scanner source to process.
|
char[] getCurrentTokenSource()
\\u0061bc
then it will answer abc
.char[] getRawTokenSource()
\\u0061bc
then it will answer \\u0061bc
.int getCurrentTokenStartPosition()
int getCurrentTokenEndPosition()
int getLineStart(int lineNumber)
lineNumber
- the given line numberint getLineEnd(int lineNumber)
lineNumber
- the given line numberint[] getLineEnds()
int getLineNumber(int charPosition)
charPosition
- the given character positionint getNextToken() throws InvalidInputException
ITerminalSymbols
.
Note that the actual token ID values are subject to change if new keywords were added to the language
(for instance, 'assert' is a keyword in 1.4).InvalidInputException
- in case a lexical error was detected while reading the current tokenchar[] getSource()
void resetTo(int startPosition, int endPosition)
ITerminalSymbols.TokenNameEOF
).startPosition
- the given start positionendPosition
- the given end positionvoid setSource(char[] source)
null
, this clears the source.source
- the given sourceCopyright (c) IBM Corp. and others 2000, 2010. All Rights Reserved.