| Package | Description |
|---|---|
| gate.creole.annic.apache.lucene.analysis.standard |
A grammar-based tokenizer constructed with JavaCC.
|
| Modifier and Type | Field and Description |
|---|---|
StandardTokenizerTokenManager |
StandardTokenizer.token_source |
| Modifier and Type | Method and Description |
|---|---|
void |
StandardTokenizer.ReInit(StandardTokenizerTokenManager tm) |
| Constructor and Description |
|---|
StandardTokenizer(StandardTokenizerTokenManager tm) |
Copyright © 2024 GATE. All rights reserved.