| Package | Description |
|---|---|
| gate.creole.annic.apache.lucene.analysis.standard |
A grammar-based tokenizer constructed with JavaCC.
|
| Modifier and Type | Method and Description |
|---|---|
ParseException |
StandardTokenizer.generateParseException() |
| Modifier and Type | Method and Description |
|---|---|
Token |
StandardTokenizer.next()
Returns the next token in the stream, or null at EOS.
|
Copyright © 2024 GATE. All rights reserved.