com.exalead.mot.components

Class SubTokenizer

java.lang.Object

com.exalead.gni.GNIObject

com.exalead.mot.core.Processor

com.exalead.mot.components.SubTokenizer

Direct Known Subclasses:

DutchTokenizer, GermanTokenizer, NorwegianTokenizer

public abstract class SubTokenizer
extends Processor

Base subtokenizer

Field Summary

Fields

Modifier and Type	Field and Description
Normalizer	normalizer Subtokenization involves normalizing subcomponents
java.lang.String	resource The processor resource

Fields inherited from class com.exalead.mot.core.Processor

fields, name

Constructor Summary

Constructors

Constructor and Description
SubTokenizer(java.lang.String name, java.lang.String resource, java.lang.String fields, Normalizer normalizer) Initialize a new SubTokenizer

Method Summary

Modifier and Type	Method and Description
static java.lang.String	getConcatLowercaseTag()
static java.lang.String	getConcatNormalizeTag()
static java.lang.String	getLowercaseTag()
static java.lang.String	getNormalizeTag()

Methods inherited from class com.exalead.mot.core.Processor

checkResource, destroy, getName, init, init

Methods inherited from class com.exalead.gni.GNIObject

printInternalCppPointer

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

resource

public java.lang.String resource

The processor resource

normalizer

public Normalizer normalizer

Subtokenization involves normalizing subcomponents

Constructor Detail

SubTokenizer

public SubTokenizer(java.lang.String name,
                    java.lang.String resource,
                    java.lang.String fields,
                    Normalizer normalizer)

Initialize a new SubTokenizer

Parameters:

name - The processor name

resource - The associated resource name

fields - The list of fields on which it's active

Method Detail

getConcatLowercaseTag

public static java.lang.String getConcatLowercaseTag()

getConcatNormalizeTag

public static java.lang.String getConcatNormalizeTag()

getLowercaseTag

public static java.lang.String getLowercaseTag()

getNormalizeTag

public static java.lang.String getNormalizeTag()