com.exalead.mercury.mami.crawl.v21

Class CustomCrawlConfig

java.lang.Object

com.exalead.mercury.mami.crawl.v21.CustomCrawlConfig

All Implemented Interfaces:

com.exalead.util.Checkable, java.io.Serializable, java.lang.Cloneable

public class CustomCrawlConfig
extends java.lang.Object
implements com.exalead.util.Checkable, java.io.Serializable, Cloneable

Custom processors specification.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
java.lang.String	crawlerTemplate
java.lang.String	fetcherClassId
java.lang.String	htmlProcessorClassId
java.lang.String	linksFilterClassId
java.lang.String	postProcessorClassId
java.lang.String	preProcessorClassId
java.lang.String	processorClassId

Constructor Summary

Constructors

Constructor and Description
CustomCrawlConfig()
CustomCrawlConfig(CustomCrawlConfig o) Copy constructor

Method Summary

Modifier and Type	Method and Description
void	check(boolean deep, java.lang.String errorContext) Checks this CustomCrawlConfig.
CustomCrawlConfig	clone() Creates and returns a deep copy of this CustomCrawlConfig.
static CustomCrawlConfig	fromString(java.lang.String s) String representation of this CustomCrawlConfig.
java.lang.String	getCrawlerTemplate() Alternatively, specify the url of a xml file describing the whole crawler.
java.lang.String	getFetcherClassId() Custom Fetcher.
java.lang.String	getHtmlProcessorClassId() Custom HTML Processor. Called at the of the html process pipe.
java.lang.String	getLinksFilterClassId() Custom LinksFilter. Called at the end of the links filter list.
java.lang.String	getPostProcessorClassId() Custom PostProcessor. Called at the end of the postprocess pipe.
java.lang.String	getPreProcessorClassId() Custom PreProcessor. Called at the end of the preprocess pipe.
java.lang.String	getProcessorClassId() Custom Processor. Called at the end of the process pipe.
CustomCrawlConfig	makeCopy() Creates and returns a deep copy of this CustomCrawlConfig.
static CustomCrawlConfig	readFrom(java.io.InputStream is) Read this CustomCrawlConfig from an XML fragment.
void	setCrawlerTemplate(java.lang.String crawlerTemplate) Alternatively, specify the url of a xml file describing the whole crawler.
void	setFetcherClassId(java.lang.String fetcherClassId) Custom Fetcher.
void	setHtmlProcessorClassId(java.lang.String htmlProcessorClassId) Custom HTML Processor. Called at the of the html process pipe.
void	setLinksFilterClassId(java.lang.String linksFilterClassId) Custom LinksFilter. Called at the end of the links filter list.
void	setPostProcessorClassId(java.lang.String postProcessorClassId) Custom PostProcessor. Called at the end of the postprocess pipe.
void	setPreProcessorClassId(java.lang.String preProcessorClassId) Custom PreProcessor. Called at the end of the preprocess pipe.
void	setProcessorClassId(java.lang.String processorClassId) Custom Processor. Called at the end of the process pipe.
java.lang.String	toString() String representation of this CustomCrawlConfig.
CustomCrawlConfig	withCrawlerTemplate(java.lang.String crawlerTemplate)
CustomCrawlConfig	withFetcherClassId(java.lang.String fetcherClassId)
CustomCrawlConfig	withHtmlProcessorClassId(java.lang.String htmlProcessorClassId)
CustomCrawlConfig	withLinksFilterClassId(java.lang.String linksFilterClassId)
CustomCrawlConfig	withPostProcessorClassId(java.lang.String postProcessorClassId)
CustomCrawlConfig	withPreProcessorClassId(java.lang.String preProcessorClassId)
CustomCrawlConfig	withProcessorClassId(java.lang.String processorClassId)
void	writeTo(java.io.OutputStream os) Write this CustomCrawlConfig as an XML fragment

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

preProcessorClassId

public java.lang.String preProcessorClassId

fetcherClassId

public java.lang.String fetcherClassId

processorClassId

public java.lang.String processorClassId

htmlProcessorClassId

public java.lang.String htmlProcessorClassId

linksFilterClassId

public java.lang.String linksFilterClassId

postProcessorClassId

public java.lang.String postProcessorClassId

crawlerTemplate

public java.lang.String crawlerTemplate

Constructor Detail

CustomCrawlConfig

public CustomCrawlConfig()

CustomCrawlConfig

public CustomCrawlConfig(CustomCrawlConfig o)

Copy constructor

Method Detail

setPreProcessorClassId

public void setPreProcessorClassId(java.lang.String preProcessorClassId)

Custom PreProcessor.
Called at the end of the preprocess pipe.

getPreProcessorClassId

public java.lang.String getPreProcessorClassId()

Custom PreProcessor.
Called at the end of the preprocess pipe.

withPreProcessorClassId

public CustomCrawlConfig withPreProcessorClassId(java.lang.String preProcessorClassId)

setFetcherClassId

public void setFetcherClassId(java.lang.String fetcherClassId)

Custom Fetcher.

getFetcherClassId

public java.lang.String getFetcherClassId()

Custom Fetcher.

withFetcherClassId

public CustomCrawlConfig withFetcherClassId(java.lang.String fetcherClassId)

setProcessorClassId

public void setProcessorClassId(java.lang.String processorClassId)

Custom Processor.
Called at the end of the process pipe. Catches all mime types.

getProcessorClassId

public java.lang.String getProcessorClassId()

Custom Processor.
Called at the end of the process pipe. Catches all mime types.

withProcessorClassId

public CustomCrawlConfig withProcessorClassId(java.lang.String processorClassId)

setHtmlProcessorClassId

public void setHtmlProcessorClassId(java.lang.String htmlProcessorClassId)

Custom HTML Processor.
Called at the of the html process pipe. Catches only html documents.

getHtmlProcessorClassId

public java.lang.String getHtmlProcessorClassId()

Custom HTML Processor.
Called at the of the html process pipe. Catches only html documents.

withHtmlProcessorClassId

public CustomCrawlConfig withHtmlProcessorClassId(java.lang.String htmlProcessorClassId)

setLinksFilterClassId

public void setLinksFilterClassId(java.lang.String linksFilterClassId)

Custom LinksFilter.
Called at the end of the links filter list. Can decide whether to crawl an outgoing link.

getLinksFilterClassId

public java.lang.String getLinksFilterClassId()

Custom LinksFilter.
Called at the end of the links filter list. Can decide whether to crawl an outgoing link.

withLinksFilterClassId

public CustomCrawlConfig withLinksFilterClassId(java.lang.String linksFilterClassId)

setPostProcessorClassId

public void setPostProcessorClassId(java.lang.String postProcessorClassId)

Custom PostProcessor.
Called at the end of the postprocess pipe.

getPostProcessorClassId

public java.lang.String getPostProcessorClassId()

Custom PostProcessor.
Called at the end of the postprocess pipe.

withPostProcessorClassId

public CustomCrawlConfig withPostProcessorClassId(java.lang.String postProcessorClassId)

setCrawlerTemplate

public void setCrawlerTemplate(java.lang.String crawlerTemplate)

Alternatively, specify the url of a xml file describing the whole crawler.

getCrawlerTemplate

public java.lang.String getCrawlerTemplate()

Alternatively, specify the url of a xml file describing the whole crawler.

withCrawlerTemplate

public CustomCrawlConfig withCrawlerTemplate(java.lang.String crawlerTemplate)

clone

public CustomCrawlConfig clone()

Creates and returns a deep copy of this CustomCrawlConfig.

Overrides:

clone in class java.lang.Object

makeCopy

public CustomCrawlConfig makeCopy()

Creates and returns a deep copy of this CustomCrawlConfig.

readFrom

public static CustomCrawlConfig readFrom(java.io.InputStream is)
                                  throws javax.xml.bind.JAXBException

Read this CustomCrawlConfig from an XML fragment.

Throws:

javax.xml.bind.JAXBException

writeTo

public void writeTo(java.io.OutputStream os)
             throws javax.xml.bind.JAXBException,
                    java.io.IOException

Write this CustomCrawlConfig as an XML fragment

Throws:

javax.xml.bind.JAXBException

java.io.IOException

fromString

public static CustomCrawlConfig fromString(java.lang.String s)
                                    throws javax.xml.bind.JAXBException,
                                           java.io.UnsupportedEncodingException

String representation of this CustomCrawlConfig.

Throws:

javax.xml.bind.JAXBException

java.io.UnsupportedEncodingException

toString

public java.lang.String toString()

String representation of this CustomCrawlConfig.

Overrides:

toString in class java.lang.Object

check

public void check(boolean deep,
                  java.lang.String errorContext)
           throws com.exalead.util.TypedException

Checks this CustomCrawlConfig.

Specified by:

check in interface com.exalead.util.Checkable

Throws:

com.exalead.util.TypedException