Clone index does not keep "number_of_replicas" setting, nor it respects index templates

[pmishev]

Elasticsearch version (bin/elasticsearch --version): 7.12.1

Description of the problem including expected versus actual behavior:

There could as well be 2 unrelated problems:

1. When cloning an index, number_of_replicas of the target index is not kept the same as the source index.

2. If there is an index template that sets the number_of_replicas to 0, this is being ignored as well, although the target index name matches the template pattern

Steps to reproduce:

PUT _index_template/template_1
{
  "index_patterns": ["mytest*"],
  "template": {
    "settings": {
      "number_of_replicas" : 0
    }
  }
}
PUT mytest1
GET mytest1

Observe that number_of_replicas = 0, as expected.

PUT /mytest1/_settings
{
  "settings": {
    "index.blocks.write": true
  }
}
POST /mytest1/_clone/mytest2
GET mytest2

Observe that number_of_replicas = 1, so the value was neither copied over from the source index, nor the index template was respected.

[DaveCTurner]

This behaviour is by design, and the note in the docs that says "Cloning doesn’t copy index metadata..." is intended to describe it, but I agree it could be clearer. Especially we should note that index templates have no impact on cloning. Marking this as a docs bug.

[elasticmachine]

Pinging @elastic/es-docs (Team:Docs)

[elasticmachine]

Pinging @elastic/es-core-features (Team:Core/Features)

[pmishev]

In the PR you did you added Notes the clone API doesn't copy index settings., but I don't think that's the case. All other index settings I had in my index were copied over (e.g analysis, max_ngram_diff, blocks, etc.).

At the moment I'm handling copying the number_of_replicas manually app-side, but I can't be sure if there aren't any other settings that are not being copied over as well. Perhaps you should make it more explicit exactly what is copied and what is not.

[DaveCTurner]

Sorry, it seems I've misread the code, I have just discovered a whole load of complications and it seems to vary by version. Let me investigate further what it's supposed to be doing.

[DaveCTurner]

Ok sorry for the confusion, as per #28347 these days we apparently do copy most of the settings over. The only ones that look to be overridden are index.number_of_replicas and index.auto_expand_replicas:

elasticsearch/server/src/main/java/org/elasticsearch/cluster/metadata/MetadataCreateIndexService.java

Lines 794 to 799 in 393ab2d

	if (INDEX_NUMBER_OF_REPLICAS_SETTING.exists(indexSettingsBuilder) == false) {
	indexSettingsBuilder.put(SETTING_NUMBER_OF_REPLICAS, INDEX_NUMBER_OF_REPLICAS_SETTING.get(settings));
	}
	if (settings.get(SETTING_AUTO_EXPAND_REPLICAS) != null && indexSettingsBuilder.get(SETTING_AUTO_EXPAND_REPLICAS) == null) {
	indexSettingsBuilder.put(SETTING_AUTO_EXPAND_REPLICAS, settings.get(SETTING_AUTO_EXPAND_REPLICAS));
	}

There's a bunch of other private settings that are also not being copied, but that's deliberate and they don't matter to end-users anyway.

I'll correct the docs PR that James made.

[jrodewig]

@DaveCTurner closed this with #74175. Thanks again!

[DaveCTurner]

I am not sure it's deliberate that clones always have exactly one replica by default, at least I find that a bit surprising given that other settings are copied over, and I think we might want to consider this a bug. I opened #74615 to discuss whether we want to do anything further on this front. I didn't reopen this thread since I didn't want the discussion to dwell on the point about index templates raised here.