Collapse search results using multiple criteria, add aggregations
• SQL-like "Group By" that allows for nested group criteria
• Sort results before grouping, independently of global result sorting to control top N hits
• Compute aggregation functions per-group
• Concatenate values from grouped hits to access it on the top N hits
• For more info: see "Collapsing Search Results" in the CloudView Technical Reference guide.
Data processing usability
• Test analysis pipelines using a sample document retrieved from document cache or fetched from the source.
• See how document metas, chunks and annotations are created, deleted or modified with each processing step in the pipeline.
• For more info: see "Understanding how a document meta is processed" in the CloudView Getting Started guide and "Tracking changes in a document" in the CloudView Technical Reference guide.
• More visibility: "Advanced Processing" is now available directly from the main navigation menu as "Data Processing".
• Group document processors and set a shared processing condition using the Document Processor Group processor.
• Document processors with conditions are now identified with an icon.
Secured suggest
• Can apply security so users only see suggests for documents they have access to
• Add security tokens to suggest dictionaries
• For more info: see "Creating a Suggest Dictionary" in the CloudView Technical Reference guide.
Suggest dispatcher
• Can now target multiple default suggest and add prefixes to suggestions.
• For more info: see "Setting a Suggest Dispatcher to Use Several Suggests in a Row" in the CloudView Technical Reference guide.
Administer users from the Admin Console
• Use security source to control access to administration interfaces from the "Users" screen.
• Associate roles to users and security tokens to restrict access to section of the Admin Console and Business Console.
• For more info: see "User Management" in the CloudView Administration guide.
Other Improvements
Platform
• Indexing throughput increased up to 30% by analyzing and importing in parallel (#19532).
• Data compression can be enabled on data model properties with index fields. Be careful though as compression consumes CPU and should in most cases not be used on RAM-based fields (#19733).
• Geographical facets domains have an identifier that can be used for refining instead of the whole geometry, to avoid reaching maximum URL size (#16878).
• Named entities extraction has been improved with updated ontologies and rules. In addition, named entities classes now follow data model from schema.org (#17488, #15400).
• Highlighting can be disabled on summarized fields (#18063).
• CJK support on Ontology Matcher (#17758).
• "Meta" security source has been renamed to "Security Source Dispatcher" to avoid confusion and be consistent with components such as "Suggest Dispatcher" (#19755).
• Switch to Java7 for better security and better performance (#15742).
• "Metadata" triggering condition on analysis pipelines (#9745).
• All ulimit parameters can be defined for each process in DeploymentInternal.xml (#20160).
Administration user interfaces
• Authentication to API Console is now role based like other part of the administration interfaces (#17865).
• Finer-grained control of what's generated automatically by the data model. Configure data model expansion per class and property to control on which search logic to generate facet, on which analysis pipeline to generate processors etc (#19230).
• Explicit, Fixed and Dynamic numerical ranges facets have been merged into a single "Numerical ranges" type in the UI (#20150).
Documentation
• New chapters on Configuration Management, Troubleshooting, and Performance Considerations in the Technical Reference guide
• Way more details on Named Entities in the Technical Reference guide
• More procedures for maintaining a production CloudView in the Administration Guide.
• New section that covers modifying the analysis pipeline in the Getting Started Guide.
• Updated Connector Programmer Guide.
Mashup Builder
• "Deployment" screen has been removed as its options (for associating Access-API with a Mashup-UI) are also available in the more global "Roles" screen of the Admin Console (#20274).
• Mashup plugins can be installed offline using command line tools (#15783) and are automatically retrieved by slave hosts from the master upon start (#19352).
• Bubble charts, range charts widgets (#18869, #12797).
• Configure size of the exported PNG image in the export widget (#18401).
• Trigger title can be set just like widgets to improve navigation in long lists of triggers (#17563).
• Flags for escaping Javascript and HTML (#15767).
Business Console
• To test different ontology modifications, you can now disable forms in the ontology editor instead of deleting them (#19214).
• "Relevance Tuning" has been renamed into "Boosted Facets" to be consistent with "Boosted Hits" (#20101).
Connectors
• Added "Log" connector for indexing structured log files. The connector supports Log4j, Apache and user defined formats and provides incremental indexing based on log time-stamps (#19228).
• Added "Numerical" column processor to JDBC connector to support any numerical format, including scientific notation and precision selection (#18899).
APIs
• Search API debug option for hiding ellql from query result (#20258).
• Search API debug options can be finely tuned using a combination of: all,query, fh, ph, ast, synthesis, keywords , ellql and "minus" signs.
o For instance use "d=-ellql" to remove ellql from query results
• The Storage Service "setMany" function allows setting multiple values at once (#19810).
o Usage: setMany(document, field, values)
• PAPI addDocument is no longer blocking in RAM indexing mode (without taskqueue).
• PAPI updateDocument now retrieves the document to update from the document cache and combines it with modified values to re-index the updated document (#20004).
• Several keywords can be associated to Mashup widgets to make them easier to find in the widget library (#16387).
• The “transformHighlightAndNodes” attribute of the HitConfig is replaced by the “relaxAndNodesPolicy” attribute of the SnippetOperation and HighlightOperation, which means the control of how a query is transformed for summary / highlight is now finer-grained (#18868).
Monitoring
• Added probes for dictionary status: per language term details, dictionary size, size and entries of words, n-grams, phonemes and related terms (#18101).
• Added probes for indexing latency (#8629).
Upgrade notes
Upgrade procedures
Migrating a data directory from any V6 release is supported. Don't forget to backup your data directory before starting the migration process.
Migration notes
• Changes of schema for named entities is only for installation from scratch. Migrated projects won't be affected, they will only benefit from the updates of the extracted forms.
• Deprecated options have been removed from DeploymentInternal.xml
o collapseIndexBuilders / collapseAnalysis: no sense since new V6R2014 Index Builder
o noAttributesCompression, javaMapper, javaReducer and index6: deprecated for a while
Changes that can break things
Platform
• HA processes now only communicate with other processes on the local host as long as they are available.
Mashup Builder / JSP Tags
• Switched to Servlet API 3.0. This means that the Mashup Builder war will only work on application server supporting this version of the API. If you're not using Jetty provided with CloudView (Jetty), you need to make sure your application server supports Servlet API 3.0 or update it accordingly (#20029).
Connectors
• When a scheduled connector scan is in progress, any new scan request will be queued. Previously new requests were just lost. Scan queues are not saved on disk for now, which means it will be lost on product restart (#19154).
Major fixed issues
Ticket
Summary
#20285
Taskqueue might get corrupted when restarting the indexing server during import
#13972
Task scheduler triggers job instantly in certain conditions
#14972
[Mashup HA] No more mashup-builder if one of the searchservers is down
#19141
[Mashup HA] Local search target doesn't work as expected in mashup builder
#19633
[Mashup HA] Mashup failover can redirect a query to a "non up-to-date" empty index
#20355
[Mashup HA] Access API does not answer if a searchserver goes out of memory (java heap space)
#19227
MashupAjaxClient can't work if the application uses url rewrite
#19294
Bad handling of proximity for prefix handler targeting multiple fields
#19931
Bad proximity computed for multi-valued alphanum fields
#19997
Failed to build dictionary on Windows 2012 after file deletion fail
#20012
Conditional display trigger doesn't work with subwidgets
#20042
CSV connector doesn't handle double-quotes
#13068
Answer group completely "excluded" is not returned in search answer
#14297
Fix the polygon edition in Gmap widget to keep "edit state"
#17378
Return No Thumbnail Icon when format is not supported (ex: TIFF)
#17701
In ontology matcher, concatAlphaNum=false breaks matching
#18197
Hostagent process doesn't log in NGRUNDIR directory
#19851
Cannot rotate convert logs on windows
#19992
clearSliceReplica doesn't work if index slices are not collapsed
#20120
Exclude refinement on value facet returns hierarchical facet
#20226
Explicit geo facets lose their title when excluded
#20250
Monitoring console is broken on Internet Explorer due to javascript error
