com.exalead.pdoc.analysis

Class JavaCustomSemanticProcessor

java.lang.Object

com.exalead.pdoc.analysis.CustomDocumentProcessor

com.exalead.pdoc.analysis.JavaCustomSemanticProcessor

All Implemented Interfaces:

CVComponent, DocumentProcessor

public abstract class JavaCustomSemanticProcessor
extends CustomDocumentProcessor

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	JavaCustomSemanticProcessor.InvalidAnnotationException Exception thrown when a malformed annotation is pushed to the semantic pipe.
static class	JavaCustomSemanticProcessor.InvalidTokenException Exception thrown when a malformed token is pushed to the semantic pipe.

Field Summary

Fields

Modifier and Type	Field and Description
protected java.util.HashSet<java.lang.String>	contexts
protected java.util.HashSet<java.lang.Integer>	languages

Constructor Summary

Constructors

Constructor and Description
JavaCustomSemanticProcessor(CVComponentConfig config)

Method Summary

Modifier and Type	Method and Description
protected void	addDocumentAnnotation(Annotation annotation) Attach an annotation to the currently processed document
abstract java.lang.String[]	declareAnnotations() Called at initialization to retrieve the annotation tags that are planned to be used during processing.
abstract void	endDocument() Called when there is no more input to process in the current document.
protected AnnotatedToken	getNextToken() Pump the next token from the input stream.
Referential	getReferential()
void	init(MOTPipe p)
protected Annotation	newAnnotation(java.lang.String tag, java.lang.String displayForm, int nbTokens) Allocate a new annotation with the provided tag, value and length.
abstract void	newDocument() Called when a new document is about to get processed.
void	process(DocumentProcessingContext dContext, ProcessableDocument doc) Process a document.
abstract void	processChunk(java.lang.String chunk, int language, java.lang.String context) Called when a new input chunk is to be processed.
protected void	pushToken(AnnotatedToken token) Send a token to the output stream.
void	release() Called at end of analysis job
void	setInputContexts(java.util.Collection<java.lang.String> c)

Methods inherited from class com.exalead.pdoc.analysis.CustomDocumentProcessor

getCondition, init, setCondition

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

languages

protected java.util.HashSet<java.lang.Integer> languages

contexts

protected java.util.HashSet<java.lang.String> contexts

Constructor Detail

JavaCustomSemanticProcessor

public JavaCustomSemanticProcessor(CVComponentConfig config)
                            throws java.lang.Exception

Throws:

java.lang.Exception

Method Detail

newDocument

public abstract void newDocument()

Called when a new document is about to get processed.

endDocument

public abstract void endDocument()

Called when there is no more input to process in the current document. This is the last chance to attach annotations to the document if needed.

declareAnnotations

public abstract java.lang.String[] declareAnnotations()

Called at initialization to retrieve the annotation tags that are planned to be used during processing. Only declared annotations will be accessible on tokens retrieved with getNextToken().

Returns:

the list of all annotation tags needed or null if none

processChunk

public abstract void processChunk(java.lang.String chunk,
                                  int language,
                                  java.lang.String context)
                           throws java.lang.Exception

Called when a new input chunk is to be processed. The processor must pump tokens from pipe using getNextToken() and return them once processed to the pipe with pushToken().

Parameters:

text - the chunk text

language - the chunk language

context - the chunk context

Throws:

java.lang.Exception

See Also:

getNextToken(), newAnnotation(), pushToken()

getNextToken

protected final AnnotatedToken getNextToken()

Pump the next token from the input stream.

Returns:

the next token from the stream or null if end of input is reached

newAnnotation

protected final Annotation newAnnotation(java.lang.String tag,
                                         java.lang.String displayForm,
                                         int nbTokens)
                                  throws JavaCustomSemanticProcessor.InvalidAnnotationException

Allocate a new annotation with the provided tag, value and length. The annotation is either created or recycled from a previous use.

Parameters:

tag - the new annotation tag

value - the new annotation value

nbTokens - the new annotation length

Returns:

a fresh or recycled annotation

Throws:

JavaCustomSemanticProcessor.InvalidAnnotationException

pushToken

protected final void pushToken(AnnotatedToken token)
                        throws JavaCustomSemanticProcessor.InvalidTokenException

Send a token to the output stream. - validity of the token is checked - the token is added to the output buffer - if needed, the output buffer is flushed - the token is recycled In all cases, the token and its annotations are not usable anymore after the call.

Parameters:

token - A token allocated through a call to newToken()

Throws:

JavaCustomSemanticProcessor.InvalidTokenException

See Also:

newToken(), newAnnotation()

addDocumentAnnotation

protected final void addDocumentAnnotation(Annotation annotation)
                                    throws JavaCustomSemanticProcessor.InvalidAnnotationException

Attach an annotation to the currently processed document

Parameters:

annotation - the annotation to attach

Throws:

JavaCustomSemanticProcessor.InvalidAnnotationException

See Also:

newAnnotation()

setInputContexts

public final void setInputContexts(java.util.Collection<java.lang.String> c)

getReferential

public final Referential getReferential()

init

public final void init(MOTPipe p)

release

public final void release()

Description copied from class: CustomDocumentProcessor

Called at end of analysis job

Specified by:

release in interface DocumentProcessor

Overrides:

release in class CustomDocumentProcessor

process

public final void process(DocumentProcessingContext dContext,
                          ProcessableDocument doc)
                   throws java.lang.Exception

Description copied from interface: DocumentProcessor

Process a document.

Parameters:

dContext - Context to perform various operations related to processing

doc - The document being processed.

Throws:

java.lang.Exception