com.exalead.papi.framework.connectors

Class GenericContainerMultiThreadedConnector<SKey extends com.exalead.papi.framework.source.adapter.Key>

java.lang.Object

com.exalead.papi.framework.connectors.Connector

com.exalead.papi.framework.connectors.GenericMultiThreadedConnector<SKey>

com.exalead.papi.framework.connectors.GenericContainerMultiThreadedConnector<SKey>

All Implemented Interfaces:

com.exalead.mercury.component.CVComponent, ConnectorStatusExceptionInterface, ConnectorInterface

Direct Known Subclasses:

ContainerMultiThreadedConnector

public abstract class GenericContainerMultiThreadedConnector<SKey extends com.exalead.papi.framework.source.adapter.Key>
extends GenericMultiThreadedConnector<SKey>

GenericContainerMultiThreadedConnector is an implementation of connector This class provides the following features : - Enumeration of Documents is multithreaded (the number of thread can be easily adjusted) - Pushing documents is multithreaded (the number of thread can be easily adjusted) - Handling containers (ZIP files, PST files ..) - All the required methods of Connector class have been implemented. You don't need to manipulate the PushAPI (It is a full abstraction of the PushAPI). The constructor of your class must have 2 parameters, the first one must be of type PushAPI, the second is your implementation of GenericContainerMultiThreadedConnectorConfig Normally your inherited class should just define a new constructor (you needn't override methods).

Nested Class Summary

Nested classes/interfaces inherited from class com.exalead.papi.framework.connectors.GenericMultiThreadedConnector

GenericMultiThreadedConnector.EnumeratesFolderDeletionHelperClient

Nested classes/interfaces inherited from class com.exalead.papi.framework.connectors.Connector

Connector.ConnectorNameThreadFactory

Field Summary

Fields

Modifier and Type	Field and Description
boolean	m_HandleContainer Handle containers

Fields inherited from class com.exalead.papi.framework.connectors.GenericMultiThreadedConnector

addDocumentOperationBuilderForFirstAdd, addDocumentOperationBuilderForNotModified, addDocumentOperationBuilderForUpdate, DEFAULT_LAST_UPDATE_CHECKPOINT_BEGIN, DEFAULT_LAST_UPDATE_CHECKPOINT_END, deletedDocumentOperationBuilder, doDelete, enumeratedPipelinesManager, isUpdate, last_update_checkpoint_begin, last_update_checkpoint_end, neverSendDeletes, processingQuota, pushFolder, sourceAdapterManager, sourceEnumerator, startKeys, syncEnumeratedError, synchronizerCallbackContainer, threadFactory

Fields inherited from class com.exalead.papi.framework.connectors.Connector

config, DEFAULT_SCAN_MODE, papi

Constructor Summary

Constructors

Constructor and Description
GenericContainerMultiThreadedConnector(GenericContainerMultiThreadedConnectorConfig config) Constructor

Method Summary

Modifier and Type	Method and Description
protected void	addRootKeyInPipelineManager(SKey root, com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> opBuilder, com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> opBuilderUpdate, com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> operationBuilderNotModified) Add root key to operation (called by initializePipelinesManager())
protected com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey>	buildAddDocumentOperationBuilderForFirstSync()
protected com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey>	buildAddDocumentOperationBuilderForUpdate() Get AddDocumentOperationBuilder
com.exalead.papi.framework.operations.container.ContainerRecursiveProcessor<SKey>	buildContainerRecursiveProcessor() Build the ContainerRecursiveProcessor responsible for handling containers.
Document	fetch(java.lang.String uri, java.lang.String[] securityTokens, boolean checkSecurityTokens) Fetch function.
static java.util.Map<java.lang.String,java.lang.String>	getConnectorMetas(ConnectorInfoRequest infoRequest) Get connector-specific metadata
abstract SKey	getKeyFromUri(java.lang.String uri) Get key from an URI

Methods inherited from class com.exalead.papi.framework.connectors.GenericMultiThreadedConnector

addDocumentOperationToEnumeratedOperationPipeline, addFolderOperationToEnumeratedOperationPipeline, addSynchronizerCallback, buildDeletedDocumentOperationBuilder, buildDeletionHelper, buildSourceAdapter, buildSourceEnumerator, buildSpecializedSourceAdapters, buildStartKeyFromConf, callBeforeDestruction, fetch, getActionToUpdate, getConnectorConfig, getCurrentPAPI, getDefaultThreadFactory, getDocumentSecurityTokens, getNbThreadsPerEnumeratedDocumentOperationPipeline, getNbThreadsPerEnumeratedFolderOperationPipeline, handleRootDeletes, init, initializePipelinesManager, isdoDelete, isPushFolder, isSynchronizeSourceEnumeratorsError, isUpdate, processActionToUpdate, registerDocInProgress, registerWorkInProgress, removeSynchronizerCallback, scan, setCurrentPAPI, setNbThreadsPerEnumeratedDocumentOperationPipeline, setNbThreadsPerEnumeratedFolderOperationPipeline, setNbThreadsPerEnumeratedOperationPipeline, setSynchronizeSourceEnumeratorsError, terminateThreads

Methods inherited from class com.exalead.papi.framework.connectors.Connector

checkAbortingOperation, checkAbortingOperation, checkDataSourceAvailability, checkSecurityTokens, getConnectorName, getEndpointBuilder, getLogger, getLogger, getScanModes, getState, getState, getStatus, getStatus, getStatusException, init, onScanAborted, onScanAborted, onScanResumed, onScanResumed, onScanSuspended, onScanSuspended, scratch, selfAbortScan, selfAbortScan, synchronizeDocuments

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Field Detail

m_HandleContainer

public boolean m_HandleContainer

Handle containers

Constructor Detail

GenericContainerMultiThreadedConnector

public GenericContainerMultiThreadedConnector(GenericContainerMultiThreadedConnectorConfig config)
                                       throws java.lang.Exception

Constructor

Throws:

java.lang.Exception

Method Detail

getKeyFromUri

public abstract SKey getKeyFromUri(java.lang.String uri)
                            throws java.io.IOException

Get key from an URI

Specified by:

getKeyFromUri in class GenericMultiThreadedConnector<SKey extends com.exalead.papi.framework.source.adapter.Key>

Parameters:

uri - the uri

Returns:

the key

Throws:

java.io.IOException

buildAddDocumentOperationBuilderForUpdate

protected com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> buildAddDocumentOperationBuilderForUpdate()

Get AddDocumentOperationBuilder

Overrides:

buildAddDocumentOperationBuilderForUpdate in class GenericMultiThreadedConnector<SKey extends com.exalead.papi.framework.source.adapter.Key>

Returns:

the operation builder

buildAddDocumentOperationBuilderForFirstSync

protected com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> buildAddDocumentOperationBuilderForFirstSync()

Overrides:

buildAddDocumentOperationBuilderForFirstSync in class GenericMultiThreadedConnector<SKey extends com.exalead.papi.framework.source.adapter.Key>

buildContainerRecursiveProcessor

public com.exalead.papi.framework.operations.container.ContainerRecursiveProcessor<SKey> buildContainerRecursiveProcessor()

Build the ContainerRecursiveProcessor responsible for handling containers.

Returns:

the ContainerRecursiveProcessor

fetch

public Document fetch(java.lang.String uri,
                      java.lang.String[] securityTokens,
                      boolean checkSecurityTokens)
               throws java.lang.Exception

Fetch function. Handle containers transparently.

Overrides:

fetch in class Connector

Parameters:

uri - document identifier to retrieve

securityTokens - Security tokens of the user querying the document

checkSecurityTokens - If true, then the connector must send the document according to the security tokens given in parameters. For example the connector must return null if the user does not have credentials enough to view it, or only send the partial content the user is allowed to access.

Returns:

requested document

Throws:

java.lang.Exception - upon error.

addRootKeyInPipelineManager

protected void addRootKeyInPipelineManager(SKey root,
                                           com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> opBuilder,
                                           com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> opBuilderUpdate,
                                           com.exalead.papi.framework.operations.builders.enumerated.EnumerableOperationBuilder<SKey> operationBuilderNotModified)
                                    throws java.lang.Exception

Add root key to operation (called by initializePipelinesManager())

Overrides:

addRootKeyInPipelineManager in class GenericMultiThreadedConnector<SKey extends com.exalead.papi.framework.source.adapter.Key>

Throws:

java.lang.Exception

getConnectorMetas

public static java.util.Map<java.lang.String,java.lang.String> getConnectorMetas(ConnectorInfoRequest infoRequest)
                                                                          throws java.lang.Exception

Get connector-specific metadata

Throws:

java.lang.Exception