gate.creole.annic.apache.lucene.analysis

Class Tokenizer

java.lang.Object

gate.creole.annic.apache.lucene.analysis.TokenStream

gate.creole.annic.apache.lucene.analysis.Tokenizer

Direct Known Subclasses:

StandardTokenizer

public abstract class Tokenizer
extends TokenStream

A Tokenizer is a TokenStream whose input is a Reader.

This is an abstract class.

Field Summary

Fields

Modifier and Type	Field and Description
protected Reader	input The text source for this Tokenizer.

Constructor Summary

Constructors

Modifier	Constructor and Description
protected	Tokenizer() Construct a tokenizer with null input.
protected	Tokenizer(Reader input) Construct a token stream processing the given input.

Method Summary

Modifier and Type	Method and Description
void	close() By default, closes the input Reader.

Methods inherited from class gate.creole.annic.apache.lucene.analysis.TokenStream

next

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

input

protected Reader input

The text source for this Tokenizer.

Constructor Detail

Tokenizer

protected Tokenizer()

Construct a tokenizer with null input.

Tokenizer

protected Tokenizer(Reader input)

Construct a token stream processing the given input.

Method Detail

close

public void close()
           throws IOException

By default, closes the input Reader.

Overrides:

close in class TokenStream

Throws:

IOException