com.exalead.search.query.linguistic

Class StemmingHandler

java.lang.Object

com.exalead.search.query.linguistic.StemmingHandler

All Implemented Interfaces:

LinguisticExpanderResource.Handler

public class StemmingHandler
extends java.lang.Object
implements LinguisticExpanderResource.Handler

Implement stemming expansion for the Linguistic Expander.
A stem is a form to which affixes can be attached. Thus the English word "productions" contains the stem "produc", to which the derivational suffix -tion is attached to form a new stem "production", to which the inflectional suffix -s is attached.
Note that a stem is the part of the word that never changes even when morphologically inflected, whilst a lemma is the base form of the verb. For example, given the word "produced", its lemma is "produce", however the stem is "produc": this is because there are words such as production.

This processor expands the query with the set of all the dictionary words that have the same non-prefixed stem.
Produces forms with formName=normalized,source=$PROCESSOR_NAME

Internal details

The MOT processor is in charge of of annotating the raw Text's alphabetic tokens with the normalized words having the same stem ( http://en.wikipedia.org/wiki/Word_stem).
It does so by finding the stem of the word, then by querying the dictionary with the regular expression "stem.*"
Snowball stemmer is employed in this handler to produce stems for most languages (http://snowball.tartarus.org/). For PL, CS, ET, SK, SL, the internal CloudView stemmer is used

The post-processor simply expands the tokenized text with these words under the NORMALIZED form.

Field Summary

Fields

Modifier and Type	Field and Description
protected static org.apache.log4j.Logger	logger

Constructor Summary

Constructors

Constructor and Description
StemmingHandler(java.lang.String cloudViewStemmerResourceDir)

Method Summary

Modifier and Type	Method and Description
LinguisticExpanderResource.PostProcessorFactory	buildPostProcessorFactory()
java.util.List<SemanticProcessor>	buildSemanticProcessor()
void	release()

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

logger

protected static final org.apache.log4j.Logger logger

Constructor Detail

StemmingHandler

public StemmingHandler(java.lang.String cloudViewStemmerResourceDir)

Method Detail

release

public void release()

Specified by:

release in interface LinguisticExpanderResource.Handler

buildSemanticProcessor

public java.util.List<SemanticProcessor> buildSemanticProcessor()

Specified by:

buildSemanticProcessor in interface LinguisticExpanderResource.Handler

Returns:

a set of SemanticProcessor to annotate the TokenizedNode

buildPostProcessorFactory

public LinguisticExpanderResource.PostProcessorFactory buildPostProcessorFactory()

Specified by:

buildPostProcessorFactory in interface LinguisticExpanderResource.Handler

Returns:

a factory of PostProcessor