Previous Release Notes : R2015x.SP2
 
R2015x.SP2
 
What's New
Other Improvements
Upgrade notes
Major fixed issues
Other fixed issues
What's New
Consolidation server
The major changes introduced in this release concern the Consolidation Server. It is now independent from the Build Group, which provides a better separation of layers for more robust and efficient processing, and easier data observability.
We have also improved the grammar used for graph traversing. It now covers more advanced use cases. For more information, see the Consolidation Server Guide > Appendix > Matching Expressions on Object Graphs > Grammar section.
The Consolidation Server Guide includes several use cases showing how to use the main features of this component (use the Coffee demo shipped with CloudView to test these use cases).
Features
• Connectors are now targeting PushAPI servers instead of Build Groups (#23178). A connector can now target a Consolidation Server as well or a Build Group.
• The post-transformation processing layer has been removed. The pre-transformation processors have been replaced by Transformation processors.
• The Forward configuration is now available in the Administration Console (#23144) and has been simplified (#23208). See the Consolidation Server Guide > Consolidation Server Behavior > Forwarding Documents to other Build Groups section.
• Provide the ability to explain which nodes have triggered the impact detection using the getReason() function (#20897)
• The grammar used by the match function has been extended and covers more use cases (#21050). Check the Consolidation Server Guide > Appendix: Processors languages > Matching expression on objects graph section.
UI
• The Consolidation UI has been reorganized to display a single consolidation config. The page is split into two tabs for better readability.
• A brand new consolidation introspection console (#22926) allows you to visualize the object graph nodes and arcs. You can click on nodes to see their payload, that is to say, the metas, parts and directives contained in the document.
attach_0292822e949256597e8c30350b7a471e
Warning: The Consolidation Server is still considered as a Beta version. The main reasons are that existing APIs and the storage format are still subject to change, which could lead to a full re-indexing in case of upgrade. Also nte that no auto migration will be possible on Groovy code; we will only provide migration procedures.
What about the SDC Connector?
• SDC benefits from various bug fixes made for the Consolidation Server.
• It is now possible to declare hierarchical type for SDC objects.
• SDC is still supported in R2015x version but might be deprecated in the next release.
Other Improvements
Connectors
• The connector Log has a new parameter to limit the maximum number of lines read per file (#23190)
• The connector name is now included in the thread name for better log readability (#20934)
• JDBC Connector: when loading the configuration page in the Administration Console, the SQL query used to display retrievable fields is only executed on demand (not each time the configuration is loaded) (#22905)
• A Java PushAPI Filter is available to customize data processing at the connector level. It supports inline or file-based java code (#19285) 
attach_7094141d0d5b61b0b3a8fb3d4fbf2a74
Platform
Document format
• BMP files with invalid file size in header are correctly converted (#23547)
Indexing
• The behavior of the triggerIndexingJob method has changed when used without Task Queue (#23154). If there is no task queue, the method may commit an indexing job if a document analysis has been started. Unlike the sync method, this method does not block the PAPI.
• "Repush from cache" action provides information about its processing (#21107), i.e., the batch size and the number of documents that have already been repushed.
• Binary fields can now be multivalued (#23240)
Search
• HTTP parameters that should not invalidate the Search API cache can now be configured (#22828). The parameters excluded by default are: cache, user_id, userquery_id, usersession_id.
• New string functions are now available for virtual expression: #strnormalize and #strlower (#21821). They might come in handy when sorting on a field containing accented characters.
• The Search API provides the same behavior for the selection of hit content, virtual fields and facets (#22948)
o include mechanism: use_logic_hit_metas, use_logic_facets, use_logic_virtual_fields
o exclude mechanism: remove_hit_meta, remove_facet, remove_virtual_field
• New "executor" debug parameter is now available to display the query executor in the search answer (#22879)
• Name of sort operation can now be defined independently from its expression (#22886)
attach_f5b66aa0bc02cb009674484356babaf7
• Dynamic date facet categories can be computed using ISO 8601 or American standard (#22045)
Suggest
• Number of results per suggest can be configured for the suggest dispatcher (#22831)
• Suggest permutation feature is available on static XML suggest and static resource suggest (#21853). For more details, see the Technical Guide > Configuring Search Queries > Search Suggest section.
Semantics
• Semantic Query Analysis can be enabled from the Administration Console (#22914).
• New sub-categories have been added to the Named Entities event type (NE.event): cultural, military, natural, political, religious, social, sport, security (#23277). For more details, see the Technical Guide > Appendix: Using Semantic Processors > Information Extraction Processors > Named Entities Matcher section.
• Date/time are supported in fast rules matchers (#19503). The supported formats for dates in rules are described in the Technical Guide > Appendix: Using Semantic Processors > Information Extraction Processors > Fast Rules Matcher (rule-based) section.
• A trust level can be associated to a form indexing. This level will be used as a weight for query expansion (#23528).  For more details, see the Technical Guide > Configuring Data Processing > Configuring Form Indexing > Set weight section.
Mashup UI
• Exceptions that occur in a widget are now fully logged (#23517)
• Enabling click-jacking protection for Mashup application (#23261). For more details, see the Installation & Administration Guide > Securing CloudView > Protecting your application from web attacks section.
• Gzip compression can be activated for Mashup UI application (#23380). For more details, see the Mashup Builder User Guide > Managing Applications > Enhancing Response Time Performances section.
Reporting
• Index size and number of index files on disk are provided in the Monitoring Console (#18421). Go to Services > Exalead > Index Details to have access to the new "Number of files" and "Size of the index" graphs.
• Number of threads and handles per processes is now monitored on Windows. (#23536)
• Static search Reporting has now a standard rotation config like other reportings (new attribute maxVersionsToKeep in SearchStaticReportingConfig) (#21013)
Tools
• Updating the CloudView license is possible through the Administration Console (#23134). The new license file is uploaded to the server and immediately taken into account.
• The CVdiag report generation can be aborted if needed (#17140)
• A diagnostic issue is generated when the Document cache option is enabled for a connector but not for the targeted build group (#21278)
• Retrieving existing checkpoints is now possible using the new MAMI GetIndexedCheckpointsList operation (#20738)
• A check and kill action has been added to ensure that no CloudView process is still running at the end of the cvinit stop command (#22592)
• You can use the cvconsole command line tool to export field dictionary with the dump-alphanum-dict command (#23158).
• New @Path annotation for CVComponent configuration elements allows you to display the file browser widget in the Administration Console (#22919)
Licensing
• nghostid can now produce long term ids for EC2 compatible environments (like Amazon WS). Go to Install&Admin guide > Product Install Directory (#21190)
Documentation
• Solving incompatibility with some security products (McAfee host intrusion prevention and/or Digital Guardian) is documented (#22502)
Upgrade notes
Consolidation
You cannot migrate automatically your consolidation configuration from R2015x.SP1 to R2015x.SP2.
Copy your configuration settings (transformation processors, aggregation processors, and consolidation options) and recreate the configuration in R2015x.SP2 entirely.
Related Terms
To migrate Related Terms from R2013x or lower versions, you must:
1. Migrate to R2014x.SP4
2. Start the product and wait until all processes are started
3. Migrate to R2015x.SP2.
Caution: The migrator needs twice the index space to migrate Related Terms. If this space is not available, the migration will crash because of disk full.
To save space, we recommend launching a full compact before migrating. To do so from the command line:
1. Go to <DATADIR>/bin
2. Launch the cvdebug tool: ./cvdebug
3. Launch the index full compact with: index compact [buildGroup] [indexDir] [slice]
4. Optionally, to also delete temporary files (.tmp) and useless slots, launch: index cleanup [buildGroup] [indexDir] [instance] [slice]
Major fixed issues
Ticket
Title
Existing since
Component
#23578
Mashup ui secures only '/page/' URLs
Prehistoric defect
360/Mashup UI & API
#23287
Do not authorize the "Use for hit content" attribute for Dynamic Date facets
V6R2015x.SP1
Administration Console
#22962
Alerting modifications should only need a "GoLive" action to be taken into account and no "Apply" action
Prehistoric defect
Alerting
#23180
Custom publisher configuration disappears
V6R2015x.SP1
Alerting
#23315
Apply configuration is too slow when config folder begins to be too big
V6R2014x.SP4
Config system / APIs
#23317
In connector configuration, JPEG is no more selected by default and Administration Console display is wrong
V6R2015x.SP1
Connectors / Connector File Systems
#22798
Post Query is not executed in the trigger base incremental synchronization mode
V6R2014x.SP4
Connectors / Connector JDBC
#22818
Disappearance of the configuration in the connector JDBC
V6R2014x.SP4
Connectors / Connector JDBC
#22972
Checkpoint query in the trigger based Incremental Mode is not working
V6R2014x.SP4
Connectors / Connector JDBC
#23393
XML connector full scans lead to random number of indexed documents
V6R2014x.SP4
Connectors / Connector XML
#22750
GDS compact failed with SDC (Windows OS)
V6R2014x.SP4
Consolidation
#22964
Crash of consolidation server while consulting introspection and pushing documents at the same time
V6R2015x
Consolidation
#23082
Incremental update is not working with owned arcs
V6R2015x.SP1
Consolidation
#23193
When clearing a source, documents having same URIs (but coming from other sources) starting by legit deleted URIs are also deleted
V6R2015x.SP1
Consolidation
#23194
When clearing a source, links created by an other source are also deleted
V6R2015x.SP1
Consolidation
#23390
Metas removed/absent in updated documents are not removed from ram based attributes
Prehistoric defect
Consolidation
#23076
Mashup-UI SSL Documentation needs an update about the certificate name
V6R2015x.SP1
Documentation
#23551
Documentation about hotfix install/unistall should be modified
V6R2015x.SP1
Documentation
#19470
Gateway is stuck with lot of thread
V6R2013x.SP3
Generic
#16811
Failed to compute licensing status after a disk full issue
V6R2013x
Index6/Build
#23004
Corruption of random files (esp. on indexing server) and/or no more log files on windows after log rotation
V6R2014x.SP1
Indexing
#18503
Analyzer is loop crashing when using basis tech module
V6R2013x.SP2
Indexing
#19112
Metas mapped with Prefix/Pattern/... mappings are considered unprocessed
V6R2013x.SP3
Indexing
#23355
Indexing Server leaks memory and comes into a state where full compact fails
V6R2015x.SP1
Indexing
#23298
Installer calls BuildGCT without Xmx and fails
Prehistoric defect
Installer
#23416
SDC migration problem from 14.SP2 to 14x.SP4
V6R2014 SP2
Migration
#23480
Get Document Status fail/bug when document don't have a stamp
V6R2014 GA
PAPI
#22878
Form Indexing config should allow to unboost expansions
Prehistoric defect
Search
#22881
Refining prevents automatic spelling correction
V6R2014x.SP3
Search
#23116
Date prefix handler uses GMT as default when query is date only
V6R2014x.SP4
Search
#23402
Enabling similarity feature generates a wrong configuration
V6R2015x.SP1
Search
#22891
SQA: mandatory policy on unused word does not seems to work anymore
V6R2015x.SP1
Semantic
#23257
Crash of Indexing Server while using a lemmatizer with language badly set
Prehistoric defect
Semantic
#23500
Similar documents feature is broken
V6R2015x.SP1
Semantic
Other fixed issues
Ticket
Title
Existing since
Component
#23073
Issues with ChartSwitcher widget (same chart type title)
V6R2014x.SP2
360/Mashup UI & API
#23130
String aggregation not properly plugged in Mashup
V6R2014x.SP3
360/Mashup UI & API
#23284
isAlive always returns "node alive" if displayStackTrace is disabled
V6R2014x.SP2
360/Mashup UI & API
#23319
Mashup Builder does not display error when apply configuration failed
V6R2015x.SP1
360/Mashup UI & API
#23330
Mashup SDK classpath is missing access-apis.jar and ehcache-core.2.4.4.jar
V6R2015x.SP1
360/Mashup UI & API
#23379
ResourceFilter should call chain.doFilter
V6R2014x.SP4
360/Mashup UI & API
#23586
Mashup UI may display full stack trace to end user even if stack display disabled
Prehistoric defect
360/Mashup UI & API
#23488
Typo in bubble chart legend description
V6R2015x.SP1
360/Mashup UI & API
#17212
Token alerting documentation and UI improvements
V6R2013x.SP1
Administration Console
#20268
"Pushed tasks" graph is hidden from the admin console in Task Queue mode
V6R2014 SP2
Administration Console
#22052
Too many incoherent Date/Time format description in UI
V6R2014x.SP2
Administration Console
#23061
Impossible to create facet due to quantic error on trusted queries
V6R2014x.SP3
Administration Console
#23192
Fix custom components widgets to be network latency proof
Prehistoric defect
Administration Console
#23007
Testing semantic resource: ontology matcher mix
V6R2014x.SP4
Business Console
#23260
Alerting UI & multiple apps
Prehistoric defect
Business Console
#23098
In semantic console, lack of current data model processors in the semantic pipeline
V6R2015x.SP1
Business Console
#19478
Add break between chunks should be disabled by default on "uri" mapping
V6R2013x.SP3
Config system / APIs
#19732
No warning saying a CloudView restart is needed when removing a Build Group
V6R2014 GA
Config system / APIs
#20452
Missing a config check to avoid multiContextField="false" to be put in advancedPipeline.getMultiContextFields()
V6R2014 GA
Config system / APIs
#23219
SolidWorks part file extension is SLDPRT (not SLDPTR)
V6R2015x.SP1
Connectors / Connector File Systems
#23268
"Never send delete" option push already indexed documents
V6R2014x.SP4
Connectors / Connector File Systems
#18099
EXALEAD Eclipse plugin does not cover XMLProcessors
V6R2013x.SP2
Connectors / Connectors Framework Java
#22487
Tooltip of connector authentication option is invalid
V6R2014x.SP3
Connectors / Data Processing
#23024
Adding a Consolidation Server should not need a product restart
V6R2015x
Consolidation
#23391
Metas removed from RAM-based attributes are not updated anymore
Prehistoric defect
Consolidation
#23114
Adding a remote root set does not work
V6R2014x.SP3
Crawler
#23183
Possible loop crashing due to race condition in DictionaryBuilderService at startup
V6R2014x
Dictionaries
#18396
No documentation about "Ranks remapping"
V6R2013x.SP2
Documentation
#20836
GDict Pruning policies documentation
V6R2014 GA
Documentation
#23175
Wrong command line to uninstall hotfix
Prehistoric defect
Documentation
#23293
Wrong password in the trustore
V6R2014x.SP3
Documentation
#22675
Crash of ss0 : /%NETHOOD%/: java.lang.NumberFormatException: NE at org.eclipse.jetty.util.TypeUtil.parseInt(TypeUtil.java:320)
V6R2014x.SP3
Generic
#23550
hotfix.sh : -datadir option cannot be placed anywhere in the command line
V6R2015x.SP1
Generic
#23225
Document Preview: the font size is different from the original document
V6R2014x.SP4
Generic
#20929
Document Processor prerequisite not working with DocumentProcessorGroup
V6R2014x
Indexing
#21854
Analysis and import states are "stopping" when CV is frozen
V6R2014x.SP2
Indexing
#23510
DataModelClassCondition does not take into account connector default class
Prehistoric defect
Indexing
#22977
JRDS does not collect any data when the crawler is not deployed
V6R2014x.SP3
Monitoring Console
#23628
After timeouts, some JRDS tasks throw NPEs
V6R2014 SP2
Monitoring Console
#23207
NullPointer exception with PAPI Encoder
V6R2014x.SP4
PAPI
#23354
ClearConnector with clearCache=false clears cache entries anyway
V6R2015x.SP1
PAPI
#21646
CSV Report file log rotation fails automatically (at least for MS Windows)
Prehistoric defect
Reporting
#19721
count of RelatedTerms on AIX is incorrect (over the Integer.MAX_VALUE)
V6R2014 GA
Search
#22219
FUZZYAND and dynamic parameter at query time
V6R2013x.SP3
Search
#23019
Slice crash in CopyNodePropertiesProcessor::processDirect with localsort when copying alphanum
V6R2014x.SP2
Search
#23187
Group by aggregation with empty values
V6R2014x
Search
#23220
NPE in MergeSearchAnswer of FederatedSearchClient
V6R2015x.SP1
Search
#23375
#strlower implementation does not support accents and special characters
V6R2015x
Search
#23382
add_virtual_field allows you to override an existing field but does not preserve the declaration order
Prehistoric defect
Search
#23445
Facet autotile - missing config check on latitudes and longitudes values
V6R2014x.SP4
Search
#23629
use_logic_hit_metas duplicates csv metas
V6R2015x.SP1
Search
#22980
Value facet refine triggers unwanted highlighting
V6R2014x.SP4
Search
#23537
Low traffic queries take at least 300ms
V6R2015x.SP1
Search
#20958
Linguistic expansions take a lot of CPU time even when almost no processing is involved
V6R2014 SP1
Semantic
#22862
Related Terms - getRelatedTerms legacy method does not seem to work
V6R2015x.SP1
Semantic
#23105
Strange sorting when using FormIndexing for Korean language
V6R2013x.SP3
Semantic
#23133
LanguageConfiguration options are lost
V6R2014 GA
Semantic