com.exalead.mercury.mami.crawl.v21

Class Feed

java.lang.Object

com.exalead.mercury.mami.crawl.v21.IRoot

com.exalead.mercury.mami.crawl.v21.Feed

All Implemented Interfaces:

com.exalead.util.Checkable, java.io.Serializable, java.lang.Cloneable

public class Feed
extends IRoot
implements com.exalead.util.Checkable, java.io.Serializable, Cloneable

A feed.
Contains KeyValue* that are mapped to metas on all documents crawled from this root.
Beware: there is a 4KB limit on the whole url + metas storage.

See Also:

Serialized Form

Field Summary

Fields

Modifier and Type	Field and Description
static boolean	DEFAULT_FIND_FEEDS
static boolean	DEFAULT_FIND_MEDIA_LINKS
static boolean	DEFAULT_FORCE_FEED_MIME_TYPE
static boolean	DEFAULT_INDEX_FEED_ITEMS
static boolean	DEFAULT_INDEX_ITEM_DOCUMENTS
static int	DEFAULT_REFRESH_PERIOD_S
boolean	findFeeds
boolean	findMediaLinks
boolean	forceFeedMimeType
boolean	indexFeedItems
boolean	indexItemDocuments
java.util.List<KeyValue>	keyValue
int	refreshPeriodS

Fields inherited from class com.exalead.mercury.mami.crawl.v21.IRoot

DEFAULT_GROUP, DEFAULT_PRIORITY, DEFAULT_SITE, group, kvs, priority, site, url

Constructor Summary

Constructors

Constructor and Description
Feed()
Feed(Feed o) Copy constructor

Method Summary

Modifier and Type	Method and Description
void	check(boolean deep, java.lang.String errorContext) Checks this Feed.
Feed	clone() Creates and returns a deep copy of this Feed.
static Feed	fromString(java.lang.String s) String representation of this Feed.
java.util.List<KeyValue>	getKeyValue()
int	getRefreshPeriodS() how often to refresh this feed, default 10min
boolean	isFindFeeds() whether to crawl feeds found in html headers
boolean	isFindMediaLinks() find and youtube/dailymotion links in item text and push them as metas
boolean	isForceFeedMimeType() force processing of url as xml feed (for servers returning buggy content types) can't work with findFeeds enabled.
boolean	isIndexFeedItems() whether to index all items found in the feed with metas, before crawling them
boolean	isIndexItemDocuments() whether to crawl the items and index the full item pages
Feed	makeCopy() Creates and returns a deep copy of this Feed.
static Feed	readFrom(java.io.InputStream is) Read this Feed from an XML fragment.
void	setFindFeeds(boolean findFeeds) whether to crawl feeds found in html headers
void	setFindMediaLinks(boolean findMediaLinks) find and youtube/dailymotion links in item text and push them as metas
void	setForceFeedMimeType(boolean forceFeedMimeType) force processing of url as xml feed (for servers returning buggy content types) can't work with findFeeds enabled.
void	setIndexFeedItems(boolean indexFeedItems) whether to index all items found in the feed with metas, before crawling them
void	setIndexItemDocuments(boolean indexItemDocuments) whether to crawl the items and index the full item pages
void	setRefreshPeriodS(int refreshPeriodS) how often to refresh this feed, default 10min
java.lang.String	toString() String representation of this Feed.
Feed	withFindFeeds(boolean findFeeds)
Feed	withFindFeeds(java.lang.Boolean findFeeds)
Feed	withFindMediaLinks(boolean findMediaLinks)
Feed	withFindMediaLinks(java.lang.Boolean findMediaLinks)
Feed	withForceFeedMimeType(boolean forceFeedMimeType)
Feed	withForceFeedMimeType(java.lang.Boolean forceFeedMimeType)
Feed	withGroup(java.lang.String group)
Feed	withIndexFeedItems(boolean indexFeedItems)
Feed	withIndexFeedItems(java.lang.Boolean indexFeedItems)
Feed	withIndexItemDocuments(boolean indexItemDocuments)
Feed	withIndexItemDocuments(java.lang.Boolean indexItemDocuments)
Feed	withKeyValue(java.util.Collection<KeyValue> __values)
Feed	withKeyValue(KeyValue... __values)
Feed	withKvs(java.lang.String kvs)
Feed	withPriority(int priority)
Feed	withPriority(java.lang.Integer priority)
Feed	withRefreshPeriodS(int refreshPeriodS)
Feed	withRefreshPeriodS(java.lang.Integer refreshPeriodS)
Feed	withSite(boolean site)
Feed	withSite(java.lang.Boolean site)
Feed	withUrl(java.lang.String url)
void	writeTo(java.io.OutputStream os) Write this Feed as an XML fragment

Methods inherited from class com.exalead.mercury.mami.crawl.v21.IRoot

getGroup, getKvs, getPriority, getUrl, isSite, setGroup, setKvs, setPriority, setSite, setUrl

Methods inherited from class java.lang.Object

equals, finalize, getClass, hashCode, notify, notifyAll, wait, wait, wait

Field Detail

keyValue

public java.util.List<KeyValue> keyValue

refreshPeriodS

public int refreshPeriodS

DEFAULT_REFRESH_PERIOD_S

public static final int DEFAULT_REFRESH_PERIOD_S

See Also:

Constant Field Values

indexFeedItems

public boolean indexFeedItems

DEFAULT_INDEX_FEED_ITEMS

public static final boolean DEFAULT_INDEX_FEED_ITEMS

See Also:

Constant Field Values

indexItemDocuments

public boolean indexItemDocuments

DEFAULT_INDEX_ITEM_DOCUMENTS

public static final boolean DEFAULT_INDEX_ITEM_DOCUMENTS

See Also:

Constant Field Values

findFeeds

public boolean findFeeds

DEFAULT_FIND_FEEDS

public static final boolean DEFAULT_FIND_FEEDS

See Also:

Constant Field Values

forceFeedMimeType

public boolean forceFeedMimeType

DEFAULT_FORCE_FEED_MIME_TYPE

public static final boolean DEFAULT_FORCE_FEED_MIME_TYPE

See Also:

Constant Field Values

findMediaLinks

public boolean findMediaLinks

DEFAULT_FIND_MEDIA_LINKS

public static final boolean DEFAULT_FIND_MEDIA_LINKS

See Also:

Constant Field Values

Constructor Detail

Feed

public Feed()

Feed

public Feed(Feed o)

Copy constructor

Method Detail

withUrl

public Feed withUrl(java.lang.String url)

Overrides:

withUrl in class IRoot

withSite

public Feed withSite(boolean site)

Overrides:

withSite in class IRoot

withSite

public Feed withSite(java.lang.Boolean site)

Overrides:

withSite in class IRoot

withPriority

public Feed withPriority(int priority)

Overrides:

withPriority in class IRoot

withPriority

public Feed withPriority(java.lang.Integer priority)

Overrides:

withPriority in class IRoot

withGroup

public Feed withGroup(java.lang.String group)

Overrides:

withGroup in class IRoot

withKvs

public Feed withKvs(java.lang.String kvs)

Overrides:

withKvs in class IRoot

getKeyValue

public java.util.List<KeyValue> getKeyValue()

withKeyValue

public Feed withKeyValue(KeyValue... __values)

withKeyValue

public Feed withKeyValue(java.util.Collection<KeyValue> __values)

setRefreshPeriodS

public void setRefreshPeriodS(int refreshPeriodS)

how often to refresh this feed, default 10min

getRefreshPeriodS

public int getRefreshPeriodS()

how often to refresh this feed, default 10min

withRefreshPeriodS

public Feed withRefreshPeriodS(int refreshPeriodS)

withRefreshPeriodS

public Feed withRefreshPeriodS(java.lang.Integer refreshPeriodS)

setIndexFeedItems

public void setIndexFeedItems(boolean indexFeedItems)

whether to index all items found in the feed with metas, before crawling them

isIndexFeedItems

public boolean isIndexFeedItems()

whether to index all items found in the feed with metas, before crawling them

withIndexFeedItems

public Feed withIndexFeedItems(boolean indexFeedItems)

withIndexFeedItems

public Feed withIndexFeedItems(java.lang.Boolean indexFeedItems)

setIndexItemDocuments

public void setIndexItemDocuments(boolean indexItemDocuments)

whether to crawl the items and index the full item pages

isIndexItemDocuments

public boolean isIndexItemDocuments()

whether to crawl the items and index the full item pages

withIndexItemDocuments

public Feed withIndexItemDocuments(boolean indexItemDocuments)

withIndexItemDocuments

public Feed withIndexItemDocuments(java.lang.Boolean indexItemDocuments)

setFindFeeds

public void setFindFeeds(boolean findFeeds)

whether to crawl feeds found in html headers

isFindFeeds

public boolean isFindFeeds()

whether to crawl feeds found in html headers

withFindFeeds

public Feed withFindFeeds(boolean findFeeds)

withFindFeeds

public Feed withFindFeeds(java.lang.Boolean findFeeds)

setForceFeedMimeType

public void setForceFeedMimeType(boolean forceFeedMimeType)

force processing of url as xml feed (for servers returning buggy content types)
can't work with findFeeds enabled.

isForceFeedMimeType

public boolean isForceFeedMimeType()

force processing of url as xml feed (for servers returning buggy content types)
can't work with findFeeds enabled.

withForceFeedMimeType

public Feed withForceFeedMimeType(boolean forceFeedMimeType)

withForceFeedMimeType

public Feed withForceFeedMimeType(java.lang.Boolean forceFeedMimeType)

setFindMediaLinks

public void setFindMediaLinks(boolean findMediaLinks)

find and youtube/dailymotion links in item text and push them as metas

isFindMediaLinks

public boolean isFindMediaLinks()

find and youtube/dailymotion links in item text and push them as metas

withFindMediaLinks

public Feed withFindMediaLinks(boolean findMediaLinks)

withFindMediaLinks

public Feed withFindMediaLinks(java.lang.Boolean findMediaLinks)

clone

public Feed clone()

Creates and returns a deep copy of this Feed.

Specified by:

clone in class IRoot

makeCopy

public Feed makeCopy()

Creates and returns a deep copy of this Feed.

Overrides:

makeCopy in class IRoot

readFrom

public static Feed readFrom(java.io.InputStream is)
                     throws javax.xml.bind.JAXBException

Read this Feed from an XML fragment.

Throws:

javax.xml.bind.JAXBException

writeTo

public void writeTo(java.io.OutputStream os)
             throws javax.xml.bind.JAXBException,
                    java.io.IOException

Write this Feed as an XML fragment

Overrides:

writeTo in class IRoot

Throws:

javax.xml.bind.JAXBException

java.io.IOException

fromString

public static Feed fromString(java.lang.String s)
                       throws javax.xml.bind.JAXBException,
                              java.io.UnsupportedEncodingException

String representation of this Feed.

Throws:

javax.xml.bind.JAXBException

java.io.UnsupportedEncodingException

toString

public java.lang.String toString()

String representation of this Feed.

Overrides:

toString in class IRoot

check

public void check(boolean deep,
                  java.lang.String errorContext)
           throws com.exalead.util.TypedException

Checks this Feed.

Specified by:

check in interface com.exalead.util.Checkable

Overrides:

check in class IRoot

Throws:

com.exalead.util.TypedException