Unloadable indices for Elasticsearch 2.3.4

[m31collision]

This commit adds unloadable "phantom" indices mainly for reducing heap usage when you need to store huge amount of data and search time does not matter.

Index consist of shards which are distributed accross nodes in the cluster. Unloading works per node only - each node has its own phantom indices manager, so entire index can be in state of partially loaded.

When index reqested to be unloadable:

• entire index goes into read-only, so data will not be changed - no need in translogs
• phantom shards replaces standard ones and comes into play
• they caches all static stats because data is read-only
• unloads indexed data (just closes Lucene's SearcherManager)
• profit!

When search request comes:

• manager unloads sufficient amount of other shards on the current node
• shard loads data again
• search can be performed

Other search requests will load phantom shards, but when limit is reached, requests will be blocked until other search requests will be finished. As a result, the more less amount of phantom shards will be permitted to load => the more requests will wait on these unloaded shards => the more time will be requied to complete search request.

Phantom indices can be turned off to index new data.

How to use:

• Index boolean setting "index.phantom" controls whether index is unloadable or not
• Cluster bytesize/percent setting "index.phantom.max_heap_size" limits a number of loaded phantom shards per node, others are unloaded
• Deletion of phantom index works despite the read-only mode

Stats affected/added

• _cat/indices has new column with +/- that shows which indices are phantom
• _cat/shards has new column with +/- that shows which shards are loaded, empty space for standard shards
• _nodes/stats has new section with phantom manager stats
• added _stats/phantom that shows phantom shards stats additionally summarized per index