com.exalead.papi.framework.connectors

Class MultiThreadedConnector

java.lang.Object

com.exalead.papi.framework.connectors.Connector

com.exalead.papi.framework.connectors.GenericMultiThreadedConnector<com.exalead.papi.framework.source.adapter.Key>

com.exalead.papi.framework.connectors.MultiThreadedConnector

All Implemented Interfaces:

com.exalead.mercury.component.CVComponent, ConnectorStatusExceptionInterface, ConnectorInterface

public abstract class MultiThreadedConnector
extends GenericMultiThreadedConnector<com.exalead.papi.framework.source.adapter.Key>

MultiThreadedConnector is an implementation of connector This class provides the following features :

• Enumeration of Documents is multithreaded (the number of thread can be adjusted)
• Pushing documents is multithreaded (the number of thread can be adjusted) The constructor of your class must have 2 parameters, the first one must be of type PushAPI, the second is your implementation of MultiThreadedConnectorConfig Normally your inherited class should just define a new constructor (you override methods)

See Also:

for a summary of methods to implement

Nested Class Summary

Nested classes/interfaces inherited from class com.exalead.papi.framework.connectors.GenericMultiThreadedConnector

GenericMultiThreadedConnector.EnumeratesFolderDeletionHelperClient

Nested classes/interfaces inherited from class com.exalead.papi.framework.connectors.Connector

Connector.ConnectorNameThreadFactory

Field Summary

Fields inherited from class com.exalead.papi.framework.connectors.GenericMultiThreadedConnector

addDocumentOperationBuilderForFirstAdd, addDocumentOperationBuilderForNotModified, addDocumentOperationBuilderForUpdate, DEFAULT_LAST_UPDATE_CHECKPOINT_BEGIN, DEFAULT_LAST_UPDATE_CHECKPOINT_END, deletedDocumentOperationBuilder, doDelete, enumeratedPipelinesManager, isUpdate, last_update_checkpoint_begin, last_update_checkpoint_end, neverSendDeletes, processingQuota, pushFolder, sourceAdapterManager, sourceEnumerator, startKeys, syncEnumeratedError, synchronizerCallbackContainer, threadFactory

Fields inherited from class com.exalead.papi.framework.connectors.Connector

config, DEFAULT_SCAN_MODE, papi

Constructor Summary

Constructors

Constructor and Description
MultiThreadedConnector(MultiThreadedConnectorConfig config)

Method Summary

Modifier and Type	Method and Description
protected com.exalead.papi.framework.source.adapter.Key	buildStartKeyFromConf(java.lang.String configKey, boolean isFolder) Call to transform Folder Name (from RootFolders) to FolderKey Override this function if the Document URI and root folder URI do not share the same logic.
com.exalead.papi.framework.source.adapter.Key	getKeyFromUri(java.lang.String uri) Get a Key, given the document URI.

Methods inherited from class com.exalead.papi.framework.connectors.GenericMultiThreadedConnector

addDocumentOperationToEnumeratedOperationPipeline, addFolderOperationToEnumeratedOperationPipeline, addRootKeyInPipelineManager, addSynchronizerCallback, buildAddDocumentOperationBuilderForFirstSync, buildAddDocumentOperationBuilderForUpdate, buildDeletedDocumentOperationBuilder, buildDeletionHelper, buildSourceAdapter, buildSourceEnumerator, buildSpecializedSourceAdapters, callBeforeDestruction, fetch, getActionToUpdate, getConnectorConfig, getCurrentPAPI, getDefaultThreadFactory, getDocumentSecurityTokens, getNbThreadsPerEnumeratedDocumentOperationPipeline, getNbThreadsPerEnumeratedFolderOperationPipeline, handleRootDeletes, init, initializePipelinesManager, isdoDelete, isPushFolder, isSynchronizeSourceEnumeratorsError, isUpdate, processActionToUpdate, registerDocInProgress, registerWorkInProgress, removeSynchronizerCallback, scan, setCurrentPAPI, setNbThreadsPerEnumeratedDocumentOperationPipeline, setNbThreadsPerEnumeratedFolderOperationPipeline, setNbThreadsPerEnumeratedOperationPipeline, setSynchronizeSourceEnumeratorsError, terminateThreads

Methods inherited from class com.exalead.papi.framework.connectors.Connector

checkAbortingOperation, checkAbortingOperation, checkDataSourceAvailability, checkSecurityTokens, fetch, getConnectorName, getEndpointBuilder, getLogger, getLogger, getScanModes, getState, getState, getStatus, getStatus, getStatusException, init, onScanAborted, onScanAborted, onScanResumed, onScanResumed, onScanSuspended, onScanSuspended, scratch, selfAbortScan, selfAbortScan, synchronizeDocuments

Methods inherited from class java.lang.Object

clone, equals, finalize, getClass, hashCode, notify, notifyAll, toString, wait, wait, wait

Constructor Detail

MultiThreadedConnector

public MultiThreadedConnector(MultiThreadedConnectorConfig config)
                       throws java.lang.Exception

Throws:

java.lang.Exception

Method Detail

buildStartKeyFromConf

protected com.exalead.papi.framework.source.adapter.Key buildStartKeyFromConf(java.lang.String configKey,
                                                                              boolean isFolder)
                                                                       throws java.lang.Exception

Description copied from class: GenericMultiThreadedConnector

Call to transform Folder Name (from RootFolders) to FolderKey Override this function if the Document URI and root folder URI do not share the same logic. Warning : can be called with null parameter when the RootFolders is empty (to manage default root path)

Overrides:

buildStartKeyFromConf in class GenericMultiThreadedConnector<com.exalead.papi.framework.source.adapter.Key>

Parameters:

configKey - The configuration root folder

Returns:

root folder key The default implementation uses getKeyFromUri(), and calls setFolder()

Throws:

java.lang.Exception

getKeyFromUri

public com.exalead.papi.framework.source.adapter.Key getKeyFromUri(java.lang.String uri)

Description copied from class: GenericMultiThreadedConnector

Get a Key, given the document URI.

Specified by:

getKeyFromUri in class GenericMultiThreadedConnector<com.exalead.papi.framework.source.adapter.Key>