com.exalead.search.query.node

Class TokenizedNode

java.lang.Object

com.exalead.search.query.node.Node

com.exalead.search.query.node.TextNode

com.exalead.search.query.node.TokenizedNode

public class TokenizedNode
extends TextNode

A chunk of tokenized text with forms. (either coming from standard linguistic expansions or custom stuff) Not usable as such for index query, needs to be transformed first

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	TokenizedNode.Form A "form" for the query of a part of a TokenizedNode. A form spans a given number of tokens from the TokenizedNode, and is made of several other tokens.
static class	TokenizedNode.Token A token, either from the query or from one of the forms

Nested classes/interfaces inherited from class com.exalead.search.query.node.TextNode

TextNode.TextProcessingOptions

Field Summary

Fields

Modifier and Type	Field and Description
java.util.List<TokenizedNode.Form>	forms Expansion results
java.lang.String	originalString

Fields inherited from class com.exalead.search.query.node.Node

indexOptions

Constructor Summary

Constructors

Constructor and Description
TokenizedNode(java.lang.String originalString)
TokenizedNode(java.lang.String originalString, TextNode.TextProcessingOptions options, IndexOptions indexOptions)

Method Summary

Modifier and Type	Method and Description
Node	accept(NodeVisitor visitor)
void	addForm(TokenizedNode.Form f) Add a form to this node.
void	addFormFromString(java.lang.String toks, int firstToken, int nbTokensSpanned, java.lang.String formName, boolean scoreMultiWordsExpressionsAsSingleWords) A helper used in tests.
void	addToken(java.lang.String value, int kind) Add a token to the list of tokens of this node
Node	cloneNode()
java.util.List<TokenizedNode.Form>	getForms()
java.util.List<TokenizedNode.Form>	getFormsStartingAtToken(int pos)
java.lang.String	getLang() Get the ISO Language code of the node
TokenizedNode.Form	getNormalizedForm(int tokenIdx)
java.lang.String	getNormalizedOrToken(int tokenIdx) Get the normalized form of a token.
int	getNormalizedOrTokenLength(int tokenIdx) Get the length of the normalized form of a token.
java.lang.String	getOriginalString() Get the original string that created this node (optional)
java.util.List<TokenizedNode.Token>	getTokens()
java.util.List<TokenizedNode.Token>	getTokensUnsafe()
boolean	isPhrase() Should this node build a sequence match
void	setLang(java.lang.String lang) Set the ISO Language code of the node
void	setPhrase(boolean isPhrase) Set whether this node should build a sequence match

Methods inherited from class com.exalead.search.query.node.TextNode

getOptions, toELLQL

Methods inherited from class com.exalead.search.query.node.Node

copyOptionsFrom, escapeStr, getHighlight, toELLQLBinary, toELLQLConstant, toELLQLNAry, toELLQLUnary, unEscapeStr

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

originalString

public java.lang.String originalString

forms

public java.util.List<TokenizedNode.Form> forms

Expansion results

Constructor Detail

TokenizedNode

public TokenizedNode(java.lang.String originalString)

TokenizedNode

public TokenizedNode(java.lang.String originalString,
                     TextNode.TextProcessingOptions options,
                     IndexOptions indexOptions)

Method Detail

cloneNode

public Node cloneNode()

Specified by:

cloneNode in class Node

getOriginalString

public java.lang.String getOriginalString()

Get the original string that created this node (optional)

getLang

public java.lang.String getLang()

Get the ISO Language code of the node

Overrides:

getLang in class TextNode

setLang

public void setLang(java.lang.String lang)

Set the ISO Language code of the node

Overrides:

setLang in class TextNode

isPhrase

public boolean isPhrase()

Should this node build a sequence match

setPhrase

public void setPhrase(boolean isPhrase)

Set whether this node should build a sequence match

getTokens

public java.util.List<TokenizedNode.Token> getTokens()

getTokensUnsafe

public java.util.List<TokenizedNode.Token> getTokensUnsafe()

getForms

public java.util.List<TokenizedNode.Form> getForms()

getFormsStartingAtToken

public java.util.List<TokenizedNode.Form> getFormsStartingAtToken(int pos)

addToken

public void addToken(java.lang.String value,
                     int kind)

Add a token to the list of tokens of this node

addFormFromString

public void addFormFromString(java.lang.String toks,
                              int firstToken,
                              int nbTokensSpanned,
                              java.lang.String formName,
                              boolean scoreMultiWordsExpressionsAsSingleWords)

A helper used in tests.

addForm

public void addForm(TokenizedNode.Form f)

Add a form to this node.

getNormalizedForm

public TokenizedNode.Form getNormalizedForm(int tokenIdx)

getNormalizedOrToken

public java.lang.String getNormalizedOrToken(int tokenIdx)

Get the normalized form of a token. Return the token value itself if not found

getNormalizedOrTokenLength

public int getNormalizedOrTokenLength(int tokenIdx)

Get the length of the normalized form of a token. Return 1 if not found, corresponding to token

accept

public Node accept(NodeVisitor visitor)
            throws QueryProcessingException

Specified by:

accept in class Node

Throws:

QueryProcessingException