com.exalead.mercury.mami.crawl.v10

Class Source

java.lang.Object

com.exalead.mercury.mami.crawl.v10.ISource

com.exalead.mercury.mami.crawl.v10.Source

All Implemented Interfaces:

java.io.Serializable, java.lang.Cloneable

public class Source
extends ISource
implements java.io.Serializable, java.lang.Cloneable

A source configuration. A source may contain KeyValues that are pushed as metas with all crawled documents.

See Also:

Serialized Form

Nested Class Summary

Nested Classes

Modifier and Type	Class and Description
static class	Source.Roots Java class for anonymous complex type.

Nested classes/interfaces inherited from class com.exalead.mercury.mami.crawl.v10.ISource

ISource.MimeTypes, ISource.QueryParameterBlacklist, ISource.Rules

Field Summary

Fields

Modifier and Type	Field and Description
protected java.lang.Boolean	indexRedirectSources
protected Source.Roots	roots
protected java.lang.Boolean	smartRefresh
protected java.lang.Integer	smartRefreshDelayS

Fields inherited from class com.exalead.mercury.mami.crawl.v10.ISource

defaultFollow, defaultFollowRoots, defaultIndex, disableConditionalGet, enableSimpleSiteCollapsing, fetcher, keyValue, mimeTypes, mimeTypesMode, name, queryParameterBlacklist, rules, simpleSiteCollapsingDepth

Constructor Summary

Constructors

Constructor and Description
Source() Creates a new Source instance.
Source(Source o) Creates a new Source instance by deeply copying a given Source instance.

Method Summary

Modifier and Type	Method and Description
Source	clone() Creates and returns a deep copy of this object.
Source.Roots	getRoots() A list of root urls.
int	getSmartRefreshDelayS() Age in milli seconds at which we begin to smart refresh old urls.
boolean	isIndexRedirectSources() Whether to index redirections with target data.
boolean	isSmartRefresh() Whether to crawl a fraction of refreshed urls.
void	setIndexRedirectSources(java.lang.Boolean value) Sets the value of the indexRedirectSources property.
void	setRoots(Source.Roots value) Sets the value of the roots property.
void	setSmartRefresh(java.lang.Boolean value) Sets the value of the smartRefresh property.
void	setSmartRefreshDelayS(java.lang.Integer value) Sets the value of the smartRefreshDelayS property.
Source	withDefaultFollow(java.lang.Boolean value)
Source	withDefaultFollowRoots(java.lang.Boolean value)
Source	withDefaultIndex(java.lang.Boolean value)
Source	withDisableConditionalGet(java.lang.Boolean value)
Source	withEnableSimpleSiteCollapsing(java.lang.Boolean value)
Source	withFetcher(java.lang.String value)
Source	withIndexRedirectSources(java.lang.Boolean value)
Source	withKeyValue(java.util.Collection<KeyValue> values)
Source	withKeyValue(KeyValue... values)
Source	withMimeTypes(ISource.MimeTypes value)
Source	withMimeTypesMode(java.lang.String value)
Source	withName(java.lang.String value)
Source	withQueryParameterBlacklist(ISource.QueryParameterBlacklist value)
Source	withRoots(Source.Roots value)
Source	withRules(ISource.Rules value)
Source	withSimpleSiteCollapsingDepth(java.lang.Long value)
Source	withSmartRefresh(java.lang.Boolean value)
Source	withSmartRefreshDelayS(java.lang.Integer value)

Methods inherited from class com.exalead.mercury.mami.crawl.v10.ISource

getFetcher, getKeyValue, getMimeTypes, getMimeTypesMode, getName, getQueryParameterBlacklist, getRules, getSimpleSiteCollapsingDepth, isDefaultFollow, isDefaultFollowRoots, isDefaultIndex, isDisableConditionalGet, isEnableSimpleSiteCollapsing, setDefaultFollow, setDefaultFollowRoots, setDefaultIndex, setDisableConditionalGet, setEnableSimpleSiteCollapsing, setFetcher, setMimeTypes, setMimeTypesMode, setName, setQueryParameterBlacklist, setRules, setSimpleSiteCollapsingDepth

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

roots

protected Source.Roots roots

smartRefresh

protected java.lang.Boolean smartRefresh

smartRefreshDelayS

protected java.lang.Integer smartRefreshDelayS

indexRedirectSources

protected java.lang.Boolean indexRedirectSources

Constructor Detail

Source

public Source()

Creates a new Source instance.

Source

public Source(Source o)

Creates a new Source instance by deeply copying a given Source instance.

Parameters:

o - The instance to copy.

Throws:

java.lang.NullPointerException - if o is null.

Method Detail

getRoots

public Source.Roots getRoots()

A list of root urls.

Returns:

possible object is Source.Roots

setRoots

public void setRoots(Source.Roots value)

Sets the value of the roots property.

Parameters:

value - allowed object is Source.Roots

isSmartRefresh

public boolean isSmartRefresh()

Whether to crawl a fraction of refreshed urls. (default: false)

Returns:

possible object is Boolean

setSmartRefresh

public void setSmartRefresh(java.lang.Boolean value)

Sets the value of the smartRefresh property.

Parameters:

value - allowed object is Boolean

getSmartRefreshDelayS

public int getSmartRefreshDelayS()

Age in milli seconds at which we begin to smart refresh old urls. (default: 3600)

Returns:

possible object is Integer

setSmartRefreshDelayS

public void setSmartRefreshDelayS(java.lang.Integer value)

Sets the value of the smartRefreshDelayS property.

Parameters:

value - allowed object is Integer

isIndexRedirectSources

public boolean isIndexRedirectSources()

Whether to index redirections with target data. (default: true)

Returns:

possible object is Boolean

setIndexRedirectSources

public void setIndexRedirectSources(java.lang.Boolean value)

Sets the value of the indexRedirectSources property.

Parameters:

value - allowed object is Boolean

withRoots

public Source withRoots(Source.Roots value)

withSmartRefresh

public Source withSmartRefresh(java.lang.Boolean value)

withSmartRefreshDelayS

public Source withSmartRefreshDelayS(java.lang.Integer value)

withIndexRedirectSources

public Source withIndexRedirectSources(java.lang.Boolean value)

withKeyValue

public Source withKeyValue(KeyValue... values)

Overrides:

withKeyValue in class ISource

withKeyValue

public Source withKeyValue(java.util.Collection<KeyValue> values)

Overrides:

withKeyValue in class ISource

withRules

public Source withRules(ISource.Rules value)

Overrides:

withRules in class ISource

withMimeTypes

public Source withMimeTypes(ISource.MimeTypes value)

Overrides:

withMimeTypes in class ISource

withQueryParameterBlacklist

public Source withQueryParameterBlacklist(ISource.QueryParameterBlacklist value)

Overrides:

withQueryParameterBlacklist in class ISource

withName

public Source withName(java.lang.String value)

Overrides:

withName in class ISource

withFetcher

public Source withFetcher(java.lang.String value)

Overrides:

withFetcher in class ISource

withDisableConditionalGet

public Source withDisableConditionalGet(java.lang.Boolean value)

Overrides:

withDisableConditionalGet in class ISource

withDefaultIndex

public Source withDefaultIndex(java.lang.Boolean value)

Overrides:

withDefaultIndex in class ISource

withDefaultFollow

public Source withDefaultFollow(java.lang.Boolean value)

Overrides:

withDefaultFollow in class ISource

withDefaultFollowRoots

public Source withDefaultFollowRoots(java.lang.Boolean value)

Overrides:

withDefaultFollowRoots in class ISource

withEnableSimpleSiteCollapsing

public Source withEnableSimpleSiteCollapsing(java.lang.Boolean value)

Overrides:

withEnableSimpleSiteCollapsing in class ISource

withSimpleSiteCollapsingDepth

public Source withSimpleSiteCollapsingDepth(java.lang.Long value)

Overrides:

withSimpleSiteCollapsingDepth in class ISource

withMimeTypesMode

public Source withMimeTypesMode(java.lang.String value)

Overrides:

withMimeTypesMode in class ISource

clone

public Source clone()

Creates and returns a deep copy of this object.

Overrides:

clone in class ISource

Returns:

A deep copy of this object.