While developing your search application, you most likely used a small corpus to test indexing. Your focus was on getting relevant results that display according to the specifications for the application.
Once you start the testing phase of your application, indexing with a real corpus, your focus turns to performance, and the sizing you need to support it.
Sizing is a complex topic with so many variables that it is impossible to provide hard & fast rules.
We can, however, explain the main sizing considerations in a project, and the type of resource they impact.
Important: This section does not replace an Exalead professional services engagement. Sizing is complex, with many factors to consider. Consult an Exalead sizing expert before undertaking sizing.
How Project Requirements Impact Sizing
This requirement
Impacts
Project description
• Do you need exhaustive search? (It guarantees that it searches and retrieves every match, even if it takes longer).
CPU
• What is an acceptable response time? Typically, for dashboards it is 4 or 5 sec; for intranet search it is in milliseconds.
CPU & RAM
Scope: what is the expected lifecycle for this app? Can this app be used for further projects?
Number of machines, or type of hardware
How many documents can be indexed?
Mainly disk space. May also impact RAM & CPU
How many data sources, and what kind?
CPU & RAM
How many users + what is the estimated QPS (Queries Per Second)?
CPU & RAM
Index freshness - every minute/every hour/every day?
CPU, RAM & disk performance
What is the expected throughput (how many incremental changes to your corpus)?
CPU, RAM & disk performance
Does it need to be an HA deployment?
Number of machines
See also the "Before going live" in the Exalead CloudView Administration Guide for a list of issues to be aware of when going into production.
Disk Requirements
See the General System Requirements in the Exalead CloudView Installation Guide.
RAM Sizing Formula
You can estimate your total RAM requirements like this:
RAM for processes + RAM for RAM-based fields + RAM for document cache.
The table below explains this in more detail.
Memory
is used for
and requires
RAM for processes
running Exalead CloudView processes.
Fixed at 8 GB
RAM for RAM-based fields
storing metas used for faceting, sorting, grouping and in virtual expressions in memory.
Highly variable: 2* (avg size of all RAM-based meta * # docs)
RAM for document cache
caching indexed documents in memory.
20% of the size of your index.
To estimate index size
• Enterprise search: 20 kb * # docs
• Analytical dashboards or eCommerce: 3 kb * # docs
To put all this together:
• For enterprise search
Total RAM = 8 GB + 2 (avg size of all RAM-based meta * # docs) + 0.2(20 kb * # docs)
• For search-based applications (analytical dashboards) or eCommerce
Total RAM = 8 GB + 2 (avg size of all RAM-based meta * # docs) + 0.2(3 kb * # docs)
To see the RAM usage per field, use the cvdebug command-line tool located in <DATADIR>/bin and start the following command: