Package | Description |
---|---|
gate.creole.annic.apache.lucene.analysis.standard |
A grammar-based tokenizer constructed with JavaCC.
|
Modifier and Type | Field and Description |
---|---|
StandardTokenizerTokenManager |
StandardTokenizer.token_source |
Modifier and Type | Method and Description |
---|---|
void |
StandardTokenizer.ReInit(StandardTokenizerTokenManager tm) |
Constructor and Description |
---|
StandardTokenizer(StandardTokenizerTokenManager tm) |
Copyright © 2024 GATE. All rights reserved.