Uses of Class
gate.creole.annic.apache.lucene.analysis.TokenStream

Packages that use TokenStream

Package	Description
gate.creole.annic.apache.lucene.analysis	API and code to convert text into indexable tokens.
gate.creole.annic.apache.lucene.analysis.standard	A grammar-based tokenizer constructed with JavaCC.
gate.creole.annic.lucene

Uses of TokenStream in gate.creole.annic.apache.lucene.analysis

Subclasses of TokenStream in gate.creole.annic.apache.lucene.analysis

Modifier and Type	Class and Description
class	Tokenizer A Tokenizer is a TokenStream whose input is a Reader.

Methods in gate.creole.annic.apache.lucene.analysis that return TokenStream

Modifier and Type	Method and Description
TokenStream	Analyzer.tokenStream(Reader reader) Deprecated. use tokenStream(String, Reader) instead.
TokenStream	Analyzer.tokenStream(String fieldName, Reader reader) Creates a TokenStream which tokenizes all the text in the provided Reader.

Uses of TokenStream in gate.creole.annic.apache.lucene.analysis.standard

Subclasses of TokenStream in gate.creole.annic.apache.lucene.analysis.standard

Modifier and Type	Class and Description
class	StandardTokenizer A grammar-based tokenizer constructed with JavaCC.

Uses of TokenStream in gate.creole.annic.lucene

Subclasses of TokenStream in gate.creole.annic.lucene

Modifier and Type	Class and Description
class	LuceneTokenizer Implementation of token stream.

Methods in gate.creole.annic.lucene that return TokenStream

Modifier and Type	Method and Description
TokenStream	LuceneAnalyzer.tokenStream(String fieldName, Reader reader) Each analyzer is required to provide implementation of this method.