The way index files are stored on disk has been changed to get lower latency at indexing time.
As of now a new index generation impacting a small number of documents generates only one file, independently of the number of index fields.
Typically, on an index of 100M documents, we can now perform small add/delete/update transactions in less than 300ms compared to 60s in release V6R2015x.SP4.
Index Dictionary
The structure used to store the dictionary of the different fields in the index has been fully rewritten to have more capabilities.
Note that here, we are talking about the index structure NOT the global dictionary which remains the same.
This structure can now handle what was called the "Advanced pattern search" / BWPatterns feature by default.
This BWPatterns structure was previously built on demand only because it had the inconvenient of being very large compared to the dictionary itself in case of long words. It was typically 10 times bigger and during compact operations was present up to 4 times on disk.
The new structure has globally the same size as the oldest dictionary without bwpatterns structure.
By default, expansions are still performed on the global dictionary.
Other Improvements
Mashup Builder
The Highcharts, JQuery and Spring third-party libraries have been updated to their new major versions. It will be useful for users developing custom widgets, for example, to support 3D charts.
Note that these updates may impact custom widgets. These are the typical problems that may occur and the way to handle them:
Library upgrade
Potential issue and solution
Spring framework upgrade from version 3.0.5 to 4.3.7
You may need to modify some extra JSP tags.
Jquery framework upgrade from 1.7.2 to 3.2.1.
• Some problems may appear due to the major version upgrade. For example, the .live() event listener has been removed.
• The jquery-migrate plugins (v3.0.0 and v1.4.1) are included in the Mashup Builder to highlight the deprecated methods used in your code.
o If your widget is based on jquery pre-1.9, you will need to include jquery-migrate-1.4.1.min.js to help you upgrading to post-1.9. Note: jquery 1.12.3 is also included in the kit (jquery-1.12.3.min.js).
o If your widget is based on jquery post-1.9, you will need to include jquery-migrate-3.0.0.min.js to help you upgrading to jquery 3.
o So the assisted process has to be done in two steps. You can do it in one phase (with no include modifications) but all the work and fixes will be done with javascript errors only.
• All includes can be modified in the Mashup Builder Application > Javascript section.
As for now, the Mashup configuration does not migrate to the new version. If you need to create applications based on the 2018x template, you will have to copy it from the new kit to the previous one. For example, to copy the "template_web" application:
1. Copy the <DATADIR>/config/360/applications/template_web directory from the 2018x kit to the previous one (2016x) and rename it, for example, as "template_web_2018x" to avoid overriding the previous one.
2. Update the <DATADIR>/config/360/ApplicationsList.xml file by adding these lines:
The protocol used by the CloudView java MAMI clients is no longer SOAP but a standard POST XML to avoid conflicts between SOAP libraries on client side
The APIs remain unchanged and the behavior should be completely equivalent but there may be edge cases where differences may appear.
SOAP usage for custom MAMI clients is still possible for but may be deprecated in 2019x.
Sorting in SearchAPI
Case sensitivity has been redesigned and is no longer a parameter. It is now specified with expressions, to be able to separate lowercase and normalized sorting (#22952)
For example, s.myfirstsort.expr=#strlower(document_alphanum1)&s.mysecondsort.expr=#strnormalize(document_alphanum2)&s=asc(myfirstsort),desc(mysecondsort)
Meta names in SearchAPI when using use_logic_facets=false
Meta names were prefixed by the class name when the use_logic_facets Search API parameter was set to false, whereas they were not prefixed when facets were returned (#23952)
Now the behavior is the same and they are not prefixed by the class name, whatever the use_logic_facets setting.
Removal of autoWildcard feature in PrefixHandler
The behavior of this feature corresponded to a very precise use case and was misleading.
We now propose to write a CustomPrefixHandler specific to your use case. A sample will be provided.
Warm-up queries configuration
Prior to V6R2018x, warm-up queries were supposed to be URL-encoded. However, due to a bug, queries were run without URL decoding. This issue is now fixed and UTF-8 URL encoding is expected, but it can break your warm-up queries settings in the <DATADIR>/config/SearchAPI.xml file. In case of problem, make sure to provide correct url-encoded query strings.
Other changes which should be transparent and propose auto migration
Default glue parameter for OptionsComposite in widget Metrics has been changed to avoid collisions from "##" to "~~~~~" (#26337)
Upgrade notes
For the first R2018x version released in May 2017, only the configuration migration is ensured by the install.sh -migrate procedure.
The automatic data migration will be handled by a later version (most probably in July 2017) so before this release, you will need to re-index your data if any.
Recommended procedure to upgrade an application
In the following procedure, the original cvapp application was made on CloudView 2016x.R4.
1. Install a clean CloudView 2016x.R4 in the DATA <DATADIR>.
2. Deploy your app on top of it with: cvadmin apps install
3. Stop this instance.
4. Remove the use of any deprecated configuration components if any in config files (see the following subsection).
5. Untar CloudView 2018x.
6. Install CloudView 2018x using the migration mode: install.sh -migrate -data DATA
7. Start the CloudView 2018x instance.
8. Rebuild your cvapp with: cvadmin apps generate
Removal of deprecated configuration components
Components tagged as deprecated in previous releases have been removed.
They will not be removed automatically during the migration operation, so you will need to remove them from your old instance before migrating.
Impacted components are: CustomCondition, FeaturesExtractor, MLNamedEntities, QueryMatcher in the <DATADIR>/config/Analysis.xml file.