Save This Page
Home » lucene-3.0.1-src » org.apache » lucene » analysis » [javadoc | source]
public class: SinkTokenizer [javadoc | source]

All Implemented Interfaces:

Direct Known Subclasses:
    TokenRangeSinkTokenizer, TokenTypeSinkTokenizer, DateRecognizerSinkTokenizer

A SinkTokenizer can be used to cache Tokens for use in an Analyzer
Field Summary
protected  List lst     
protected  Iterator iter     
Fields inherited from org.apache.lucene.analysis.Tokenizer:
 public SinkTokenizer() 
 public SinkTokenizer(List input) 
 public SinkTokenizer(int initCap) 
Method from org.apache.lucene.analysis.SinkTokenizer Summary:
add,   close,   getTokens,   next,   reset
Methods from org.apache.lucene.analysis.Tokenizer:
close,   correctOffset,   reset
Methods from org.apache.lucene.analysis.TokenStream:
close,   end,   incrementToken,   reset
Methods from org.apache.lucene.util.AttributeSource:
addAttribute,   addAttributeImpl,   captureState,   clearAttributes,   cloneAttributes,   equals,   getAttribute,   getAttributeClassesIterator,   getAttributeFactory,   getAttributeImplsIterator,   hasAttribute,   hasAttributes,   hashCode,   restoreState,   toString
Methods from java.lang.Object:
clone,   equals,   finalize,   getClass,   hashCode,   notify,   notifyAll,   toString,   wait,   wait,   wait
Method from org.apache.lucene.analysis.SinkTokenizer Detail:
 public  void add(Token t) 
    Override this method to cache only certain tokens, or new tokens based on the old tokens.
 public  void close() throws IOException 
 public List getTokens() 
    Get the tokens in the internal List.

    WARNING: Adding tokens to this list requires the #reset() method to be called in order for them to be made available. Also, this Tokenizer does nothing to protect against java.util.ConcurrentModificationException s in the case of adds happening while #next(org.apache.lucene.analysis.Token) is being called.

    WARNING: Since this SinkTokenizer can be reset and the cached tokens made available again, do not modify them. Modify clones instead.

 public Token next(Token reusableToken) throws IOException 
    Returns the next token out of the list of cached tokens
 public  void reset() throws IOException 
    Reset the internal data structures to the start at the front of the list of tokens. Should be called if tokens were added to the list after an invocation of #next(Token)