When flattening data, it is sometimes useful to be able to generate multiple documents from a parent document. These child documents are not pushed by any source but are interesting to simplify queries performed later on the index.
We assume that previous UCs have been completed.
Step 1 - Create Child Documents from Organization with an Aggregation Processor
1. Add an aggregation processor:
a. Select Groovy as format
b. For Name, enter Organization_UC_7
c. Click Accept
2. Replace the default code by the following one:
// Process nodes having the “organization” type process("organization") { // Log the content of the document passing through this processor log.info "Child creation for organization: " + it
// Find the top country for import trade per year trades = // Get all paths to related country nodes match(it, "-isMemberOf[country].import[trade]") *.last() // fetch last node year_top = [:].withDefault() { [:].withDefault() {0} } // Big Integer def bInt = 0G; year_top_volume = [:].withDefault() { bInt }
// Build the child collection trades.each { trade -> if (year_top[trade.metas.getValue("year")]["volume"] < trade.metas.getValue("volume") .toInteger()) { year_top[trade.metas.getValue("year")]["volume"] = trade.metas.getValue("volume").toInteger(); year_top[trade.metas.getValue("year")]["country"] = trade.metas.getValue("country_id"); year_top_volume[trade.metas.getValue("year")] += trade.metas.getValue("volume").toInteger(); } } // Caution! Before pushing any new document, remove existing child documents, if any. // This operation is yielded automatically. deleteDocumentChildren(it, "/year_import/");
// create child documents year_top.each { key, value -> log.info "Year:" + key + " - " + value["volume"] + " - " + value["country"] + " - " + year_top_volume[key] ; child = createChildDocument( it, // root '/year_import/' + key, // child URI "ico_trade" // type ); // Add metas to the child document child.metas.parent_identifier = it.getUri(); child.directives.datamodel_class = "ico_trade"; // directly set with the type defined in createDocument but it can be overridden if needed child.metas.org_id = it.metas["org_id"]; child.metas.year = key; child.metas.country_id = value["country"]; child.metas.volume = value["volume"]; child.metas.globalvolume = year_top_volume[key]; yield child; } }
3. Save and apply the configuration.
Step 2 - Relaunch the Organization Aggregation and Check What Is Indexed
1. Go to the Home page.
2. Click Force aggregation and enter organization as type.
3. Open the following Mashup UI application page: http://<HOSTNAME>:<BASEPORT>/mashup-ui/page/analytics_v2
4. Select the ICO Membership tab.
You can now see the Top Import country and Global import volume for each year.
Step 3 - Change the Membership of a Country
For this operation, you need to access the server.
1. Go to the <INPUTDIR> containing the coffee sample data.
2. Change the membership of a country in the coffee database, for example, USA.
a. In your command-line tool, run sqlite3 ./coffee.db
b. Run the following commands one after the other:
delete from countries where country_id="USA"; insert into countries(country_id, name, ico_status) values ("USA", "USA", "Non Member"); .exit
Note: The insert statement adds the current timestamp to the record automatically. The JDBC connector uses it to detect this modification.
Step 4 - Rescan the Country Connector and Check What Is Indexed
1. Click Scan for the country JDBC connector.
Wait for data to be fully indexed.
2. Go to the analytics page: http://<HOSTNAME>:<BASEPORT>/mashup-ui/page/analytics_v2
3. Select the ICO Membership tab.
USA is not displayed in the Top Import country column anymore.
