Previous Release Notes : R2018x.R1
 
R2018x.R1
 
What's New
Other Improvements
Upgrade notes
Enhancements
Major fixed issues
Support Lifecycle
What's New
HDFS Support for better elasticity
On a Red Hat Linux version 7 platform, HDFS can now be configured as a backend storage.
In this mode, storage no longer needs to be local to the machine. It can be in a remote cluster.
Benefits:
Faster replication
Fast instanciation of new index on demand
Reduce probability of disk full
HA indexing possible in active/passive mode
Note: HDFS storage is still not supported by the Consolidation server.
To configure HDFS:
1. Stop your CloudView instance.
2. Go to DATADIR > config
3. Edit the DeploymentConfig.xml file.
4. For the indexing role, change the value of the storageUri attribute to:
hdfs://${HOST_HDFS}/{PATH}
5. Apply your configuration.
6. Restart your CloudView instance.
Please contact us for advice on the architecture.
Out-of-the-box Consolidation Processors & UI revamping
Several transformation and aggregation Java processors are now delivered out-of-the-box to help you cover basic use cases.
See "Consolidation Server Guide > Writing Transformation and Aggregation Processors > Java Processors > Define Java Transformation Processors > Use out-of-the-box transformation processors"
and "Consolidation Server Guide > Writing Transformation and Aggregation Processors > Java Processors > Define Java Aggregation Processors > Use out-of-the-box aggregation processors"
The UI to handle out-of-the-box and custom plugins has also been improved for a better interaction with these processors:
492453674b85328921f4d55de57be034
Define an external Push API Server
For deployment reasons, you may want to push document sources (i.e. one or more connectors, a Replay Server or a Consolidation Server) on an external Push API server located on another CloudView instance.
See "Connectors Guide > About Connectors > Defining an external Push API Server".
Sort index objects by vectorial similarity
 
The #attrsimilar function calculates similarity between a given vector and vectors in the index. For example, it can be used to detect parts with similar shape and/or size.
#attrsimilar is a query node in the index which returns all the documents matching the similarity query and calculates the similarity measure.
As it does not filter search results at all, you should combine it with a #filter to return only the documents having a similarity higher than a given threshold value.
Note: Similarity is the inverse of distance and is calculated as follows: similarity = 1 - distance
For more details about this function, see "Configuration Guide > Configuring Search Queries > Configuring and Searching Similarity Measures"
Molecule & Reaction search builtin operators
New operators have been added to perform chemical operations using an external library with chemical knowledge.
Reduce query time
Previously, inside a slice, different queries were run in parallel but a single query was using only one core.
As of now a single query can also be run on multiple cores to reduce the time perceived by the end user.
So using more slices is no longer the only solution to reduce query time.
See the description of the nbVirtualPartitionPerSlice parameter in the XML Configuration Reference.
Cumulative hotfix handling
Cumulative hotfix tracability has been improved. Timestamps are now associated to our cumulative hotfixes to easily determine which hotfix version is installed.
The cumulative hotfix version can be seen in the Administration Console > About CloudView menu, or using the following command: KITDIR/tools/hotfix list
Installing a cumulative hotfix version is now automatically understood by the system (you no longer need to uninstall the previous version manually).
Add groups of builtin and custom validators to identify CloudView Configuration issues
In the Administration Console, the Diagnostics menu indicates the number of outstanding issues detected by validators in the last applied configuration.
The Groups panel has been added to let you create your own group of configuration validators, by adding some builtin validators and your custom validators (if any).
In addition, deprecated components that are still used in the configuration are now reported in the Diagnostics.
See "Administration Guide > Monitoring CloudView > Identifying configuration issues".
Have the possibility to use an alternate JVM
For security reasons, it may be necessary to use a more recent JVM than the one provided with the kit. To do so, edit the JAVA_HOME variable in <INSTALLDIR>/ngroot.env
See "Installation Guide > Installing CloudView > Security Considerations > Use a different version of the JVM".
Other Improvements
New reference documentation included in the Online Help
The XML Configuration Reference, the Semantic Reference (under "Configuration") and the CVConsole Commands Reference (under "Administration") are now part of the Online Help and their content can be searched for.
 
Upgrade notes
The R2018x GA only proposes the migration of the configuration in the install.sh -migrate procedure.
R2018xR1 also perfoms the automatic data migration with the following restriction.
Warning: Direct migration from 2012x, 2012x SP1, 2013, 2013 SP1, 2013 SP2 to 2018x is not supported.
To migrate from an old CloudView version to the R2018x version, first migrate to R2016x.R5 then upgrade from R2016x.R5 to R2018x.
Recommended procedure to upgrade an application
In the following procedure, the original cvapp application was made on CloudView 2016x.R4.
1. Install a clean CloudView 2016x.R4 in the DATA <DATADIR>.
2. Deploy your app on top of it:
1. Go to: DATA/bin/
2. Run: ./cvadmin apps install
3. Stop this instance.
4. Remove the use of any deprecated configuration components if any in config files (see Removal of deprecated configuration components).
5. Untar CloudView 2018x.
6. Install CloudView 2018x using the migration mode: install.sh|bat -migrate -data DATA
7. Start the CloudView 2018x instance.
8. Rebuild your cvapp:
1. Go to: DATA/bin/
2. Run: ./cvadmin apps generate
Removal of deprecated configuration components
Components tagged as deprecated in previous releases have been removed.
They will not be removed automatically during the migration operation, so you will need to remove them from your old instance before migrating.
Impacted components are: CustomCondition, FeaturesExtractor, MLNamedEntities, QueryMatcher in the <DATADIR>/config/Analysis.xml file.
Migration troubleshooting
Known issue: "no such file" errors display for slices after migration of the original DATADIR
For example:
[2017/07/04-09:35:54.881] [error] [1908043086@jtp-default-1] [flis] Exception on FLIS remote call (//switchGeneration): java.lang.RuntimeException: Unable to read attr/1/slot_40-44.xml
[2017/07/04-09:35:54.881] [error] [1908043086@jtp-default-1] [flis] at com.exalead.index.XMLConfigCache.read(XMLConfigCache.java:102)
[2017/07/04-09:35:54.881] [error] [1908043086@jtp-default-1] [flis] ...
[2017/07/04-09:35:54.881] [error] [1908043086@jtp-default-1] [flis] Caused by: java.io.IOException: Cannot open files on wormfs (attr/1/slot_40-44.xml): [04] no such file
[2017/07/04-09:35:54.881] [error] [1908043086@jtp-default-1] [flis] ...
Explanation: This issue is related to the Indexing Server and the slice(s) starting time. If the Indexing Server is up and running before the slice, this issue will not occur. If the Indexing Server is not started yet and the replica tries to start with the old index generation (before migration), this type of issue will occur.
Enhancements
Ticket
Title
Component
28439
Allow to remove the facet when the aggregation has no value
360/Mashup Builder
28158
Disable sorting in Advanced Result Table widget
360/Mashup UI & API
28220
Add toString for 360 public API objects
360/Mashup UI & API
28235
Add nbClippedChildren to Access API
360/Mashup UI & API
28450
Advanced Result Table Widget: add ability to format label or value when column type is URL
360/Mashup UI & API
28460
Add default trustStore for Tomcat
360/Mashup UI & API
25006
[Admin-UI/diagnostics] Ability to activate custom diagnostics in Admin-UI
Admin-ui
26716
Improve the display and usage of custom java classes in Admin UI
Admin-ui
28030
Add tooltips to Push API Servers parameters
Admin-ui
26110
Semantic Console does not import ontology in excel format correctly
Business Console
27625
Mechanism to have in Dr CloudView all deprecated config elements used in current applied config
Config system / APIs
28318
Setup the JMX port of JRDS in Cloudview's configuration
Config system / APIs
28526
Do not restart the conso server when applying conf with changes only in the fwd rules
Config system / APIs
28541
The CSV connector does not handle record deletion.
Connectors / Connector CSV
28152
Improving indexing speed of remote compressed files in windows environment
Connectors / Connector File Systems
28185
Document how to handle filenames longer than 260 chars on Windows
Connectors / Connector File Systems
28262
Doc not up to date for hdfs drive indexing
Connectors / Connector File Systems
26954
Handle inside product "classic" transformation/aggregation processor
Consolidation
28293
Force aggregation: forward to a specific target
Consolidation
28329
Do not trigger aggregation when clearing connectors
Consolidation
28482
Warn users pushing the same document multiple times in the same transformation job
Consolidation
28100
Improve documentation on document's encoding autodetection
Convert
28680
Create convert temporary dir if not existing
Convert
28001
Report "alien" files in cvdiag
Core
28269
Be able to set an ipv6 as hostname in cloudview deployment
Core
28273
Handle DNS change without reboot all cloudview
Core
28605
Support SNI in HTTP Crawler
Crawler
19341
More doc on plugin workflow
Documentation
27573
Deliver documentation on #attrsimilar function
Documentation
27898
Improve Doc on Storage service to use consolidation server
Documentation
28565
Add link to Installation Guide on Gateway page
Documentation
28710
Add the support life cycle dates on the release notes
Documentation
27563
Optimize CID assignement
Generic
28291
Remove useless log messages "dladdr error / pointer unresolvable"
Generic
28601
Have a documented way to change the jvm
Generic
28733
More efficient compression for fixed size binary column
Generic
28513
Add a json output to geo api
Geo
23855
Support HDFS HA in WormFS
HDFS
26430
HDFS availability and scope
HDFS
27486
Build High Availability via support of shared file system
Indexing
27550
Better shrinking policy for buffer for aggregated
Indexing
27855
Replay Server should be considered as a Push API server
Indexing
28184
Indexing MAMI: GetIndexedCheckpointsList: filter checkpoints by prefix
Indexing
28226
Build FSM in a direct-to-disk way instead of using gigabytes of RAM
Indexing
28229
Document HTML Relevant Content Extractor Document processor
Indexing
28320
Itemstore import time can be largely impacted by nbVirtualClass/nbTotalColumn
Indexing
28336
Reduce compaction time and memory usage for geo field
Indexing
28455
Reduce SXI indexing contention (docs with many metas + RulesMatcher)
Indexing
28244
Allow windows service overwrite during install
Installer
28255
Improve handling of cumulative hotfix
Installer
28254
add access to the search.csv in the debug interface.
Reporting
28349
Add the output of cvdebug index dump-attribute-group-column-infos to the cvdiag
Reporting
22122
Lemmatizer expansion limiter
Search
28251
Alternative strategy for INNERJOIN when the left-hand operand is large
Search
28260
Suggest build on specific key of a dynprop can target a prefix handler without necessarily providing key
Search
28264
Virtual facet on top of RB GPS Point
Search
28371
Having parallel query over CV
Search
28379
Subsecond BOR query with 10000 terms
Search
28405
Improve builtin support of malaysian
Search
28415
Create generic attrsimilar prefix handler
Search
28493
Add a chemical filter in the index (molecule exact and substructure matching, reaction matching)
Search
28677
Improve default memory locking strategy for index files
Search
28258
Improve Attrgraph with large number of security tokens performance
Search 3D
28295
Have a log of invalid root for Expand
Search 3D
28357
Attrgraph performance improvements especially around wrap queries
Search 3D
28402
support fetch of BBOX in expand queries.
Search 3D
27390
Enhancing matching capabilities in synonym files (stop words, inverted words, separators)
Semantic
28579
Generic way to redirect the cvconsole output to a file
Tools
Major fixed issues
 
Ticket
Title
Existing since
Component
28608
Cannot remove specific key in storageService using mashup builder
Prehistoric defect
360/Mashup Builder
28175
When using Tags widget, quotation marks in a tag prevents it to being added to a query.
Prehistoric defect
360/Mashup UI & API
28358
i18n popups in debug mode doesn't appear anymore
V6R2018x
360/Mashup UI & API
28409
Widget Advanced Result Table: cannot modify condition value when changing column name
Prehistoric defect
360/Mashup UI & API
28492
DSLS Users count in mashup does not fallback on KITDIR default appslicensingconfig
V6R2016x.R4
360/Mashup UI & API
28740
AdvancedTable widget: Don't do protocol completion in case of column type is URL
V6R2018x
360/Mashup UI & API
28459
CLICorePluginPackager does not support plugin.reverse-classloading
V6R2015x
360/Plugins
28506
A user with no right in Admin Console has still access to unsafe operation
V6R2014 GA
Admin-ui
24442
Better document the Real time alerting edition workflow
V6R2014x.SP4
Alerting
28520
Crash when removing a connector.
V6R2018x
Config system / APIs
27739
FTP generates duplicated URI
Prehistoric defect
Connectors / Connector File Systems
27872
Cannot download files if they're on ftp server and file name contains spaces
Prehistoric defect
Connectors / Connector File Systems
28429
File locked by Cloudview connector on distant drive
Prehistoric defect
Connectors / Connector File Systems
28408
Native Memory Leak in the Data integ common
V6R2018x
Connectors / Connector JDBC
28054
Counters of connectors are not up to date when deployed on replay server
V6R2018x
Connectors / Connector Replay
28636
cvdebug: export-object-graph does not output all vertices
V6R2016x.R5
Consolidation
28645
Vertices may be lost during commit when the same doc is pushed many times
V6R2015x.SP4
Consolidation
28112
Long path files not removed from tmp dir when minion is closed on windows
Prehistoric defect
Convert
28399
Semaphores acquire of AsposeConverter should be timeoutless when using ThumbnailGenerator to precompute thumbnails
V6R2016x
Convert
28451
Unable to search the contents of docx file created online in SharePoint 365.
Prehistoric defect
Convert
27665
Get rid of non-trivial native thread-local variables usable in threads spawned by the JVM
Prehistoric defect
Core
28312
Allow multiple envOverride / envPrepend / envAppend on the same variable
Prehistoric defect
Core
28117
Dictionary compaction is commited only after its deletion
Prehistoric defect
Dictionaries
28212
Compact failed for FSM dict > 4G
V6R2018x
Dictionaries
28624
some dictionary entries not returned by #anumpattern
V6R2012x
Dictionaries
28215
List of search operator
V6R2016x.R4
Documentation
28443
Prereqs not anymore present in the doc
V6R2018x
Documentation
28540
"Install an add-on": missing info in EXALEAD standalone documentation
V6R2018x
Documentation
28102
DSLS not working in log debug mode
V6R2016x
Generic
28177
Allow HTTP OPTIONS method for health checks
V6R2018x
Generic
28261
strlower in a #filter can lead to slice crash when too large amount of data
Prehistoric defect
Generic
28097
indexing server crash : Assertion pending-rename is malformed
V6R2016x.R4
Indexing
28132
Full compact request ineffective
V6R2016x.R4
Indexing
28171
maxParallelFullCompacts in Indexing.xml is not respected
V6R2018x
Indexing
28423
Indexed documents are not searchable on one Front 3 minutes after the end of the replication
V6R2016x.R3
Indexing
28649
cvapps install returns OK even it fails and rollbacks to old config
Prehistoric defect
Installer
28701
Fix documentation on Light upgrade on windows when using the services
V6R2016x.R4
Installer
28343
Migration from 15x to 16x fails on slaves on windows
V6R2016x
Migration
28192
JRDS failing to initialize counters for probe CVProcessPdh with Windows not set in English
V6R2015x
Monitoring Console
28298
cvdiag coreDumps parameter is not taken into account
Prehistoric defect
Reporting
27414
MOT pool pipe is not realeased when an exception happens on Suggest init
Prehistoric defect
Search
28133
Recuring warning related to missing Source and Language facet
V6R2016x.R4
Search
28146
DynamicFullTextPrefixHandler fail to target a meta
Prehistoric defect
Search
28186
Deadlock during suffix lookup
V6R2018x
Search
28250
replication failure after ClearReplica for uid fields
V6R2018x
Search
28282
Unable to set http timeouts on SAPI SearchClient (client side)
Prehistoric defect
Search
28334
Queries with only graph leaf nodes detected as “stopwords only queries”
V6R2014x
Search
28350
AttrSimilar cannot be used with cosine function anymore
V6R2016x.R4
Search
28387
Wrong aggregation computation of multi valued doubles
V6R2016x
Search
28428
Query reporting numbers are wrong in the SAPI when using cache and disjunctive facets
Prehistoric defect
Search
28442
UQL handling of special characters: between quotes, backslashes are duplicated and wildcards deduplicated
Prehistoric defect
Search
28475
.xlsm/docm/pptm are not using Aspose for thumbnails
V6R2014x
Search
28550
Regression in preview highlight - Text overlapping on word document
V6R2016x.R5
Search
28555
NPE in fetch api : java.lang.NullPointerException: charsetName
V6R2016x.R5
Search
28594
Search server crash on access violation while refining rt cat
V6R2016x.R5
Search
28729
Leak of Java thread objects in index6 process conducting to OOM
V6R2018x
Search
28346
Cosine option for similarity search does not compute cosine similarity
V6R2016x.R3
Search 3D
28505
It is possible to retrieve arbitrary files through the exa http servers on windows when not firewalled
V6R2015x.SP3
Security
27459
Missing config check/warning with BT open source tokenizer without the addon
Prehistoric defect
Semantic
28173
SQI native XML parser doesn't support namespaces
V6R2018x
Semantic
28256
the japanese tokenizer doesn't support all cases of tokenization exceptions
Prehistoric defect
Semantic
28317
An empty ontology prevent the pipe from being initialized
V6R2015x
Semantic
28569
cvconsole command with unkown safety can't be executed with cv up
V6R2018x
Tools
Support Lifecycle
This section provides information about end-of-support dates. For more details, see http://media.3ds.com/support/DS_LifeCycleInformation.pdf.
004ef035e2e2b9caa545bb291c738875
*FULL support starts when the release is available.
*EXTENDED support starts at the end of the FULL support for one additional year when customers are current with the applicable support charges and have paid the relevant additional charge.
For more details, see https://www.3ds.com/support/policies-offers/lifecycle-policy/