Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add is-write-index flag to aliases #30942

Merged
merged 29 commits into from
Jun 15, 2018
Merged

add is-write-index flag to aliases #30942

Show file tree
Hide file tree
Changes from 8 commits
Commits
Show all changes
29 commits
Select commit Hold shift + click to select a range
5838717
add is-write-index flag to aliases
talevy May 29, 2018
7baf86b
cleanup
talevy May 31, 2018
19315ff
fix unit tests
talevy Jun 1, 2018
440fed4
update docs and tests
talevy Jun 1, 2018
135d7c1
change to validate at MetaData.build
talevy Jun 4, 2018
5bb7217
Merge branch 'master' into is-write-index-setting
talevy Jun 5, 2018
847bbe6
Merge branch 'master' into is-write-index-setting
talevy Jun 5, 2018
32ca015
add writeIndex to ToAndFromJsonMetaDataTests
talevy Jun 5, 2018
07188f3
Merge branch 'master' into is-write-index-setting
talevy Jun 12, 2018
01fab60
refactor as to not re-build aliasAndIndexLookup
talevy Jun 12, 2018
c7a3a96
migrate validation tests and setonce writeIndex
talevy Jun 12, 2018
8df9bc8
Merge branch 'master' into is-write-index-setting
talevy Jun 12, 2018
4e898c1
fix SharedClusterSnapshotRestoreIT#testRenameOnRestore
talevy Jun 12, 2018
1aac672
Merge branch 'master' into is-write-index-setting
talevy Jun 12, 2018
5ea9725
fix rest spec tests
talevy Jun 12, 2018
1c9e28f
change validation strategy and no defaulting to indexmetadata
talevy Jun 12, 2018
4755a10
update to reflect no defaulting values
talevy Jun 12, 2018
ae2b0c9
fix unit tests
talevy Jun 12, 2018
545c801
fix doc tests
talevy Jun 12, 2018
a614010
add version guard for 7.0 in IndicesAliasesRequest
talevy Jun 13, 2018
ee4142d
Merge branch 'master' into is-write-index-setting
talevy Jun 13, 2018
8c33d23
Merge branch 'master' into is-write-index-setting
talevy Jun 13, 2018
0025a51
further cleanup
talevy Jun 13, 2018
373c1e5
add randomization to AliasMetaDataTests.assertEqualInstances
talevy Jun 13, 2018
ba1162c
Merge branch 'master' into is-write-index-setting
talevy Jun 13, 2018
a1391b2
split out is_write_index check in yaml test and skip if run before 7.0
talevy Jun 13, 2018
91deba4
add test for write index swapping
talevy Jun 13, 2018
1629aef
Merge branch 'master' into is-write-index-setting
talevy Jun 14, 2018
f8dba0c
remove unused imports
talevy Jun 14, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 94 additions & 3 deletions docs/reference/indices/aliases.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -244,6 +244,94 @@ GET /alias2/_search?q=user:kimchy&routing=2,3
// CONSOLE
// TEST[continued]

[float]
[[aliases-write-index]]
==== Write Index

It is possible to associate the index pointed to by an alias as the write index.
When specified, all index and update requests against an alias that point to multiple

This comment was marked as resolved.

Sign in to view

indices will attempt to resolve to the one index that is the write index. If no such index is
found, then an exception will be thrown.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add that only one index per alias can have the flag?


It is possible to specify an index associated with an alias as a write index using both the aliases API
and index creation API.

[source,js]
--------------------------------------------------
POST /_aliases
{
"actions" : [
{
"add" : {
"index" : "test",
"alias" : "alias1",
"is_write_index" : true
}
}
]
}
--------------------------------------------------
// CONSOLE
// TEST[s/^/PUT test\n/]

In this example, we associate the alias `alias1` to both `test` and `test2`, where
`test` will be the index chosen for writing to.

[source,js]
--------------------------------------------------
PUT /alias1/_doc/1
{
"foo": "bar"
}
--------------------------------------------------
// CONSOLE
// TEST[continued]

The new document that was indexed to `/alias1/_doc/1` will be indexed as if it were
`/test/_doc/1`.

[source,js]
--------------------------------------------------
GET /test/_doc/1
--------------------------------------------------
// CONSOLE
// TEST[continued]

To swap which index is the write index for an alias, the Aliases API can be leveraged to
do an atomic swap. The swap is not dependent on the ordering of the actions.

[source,js]
--------------------------------------------------
POST /_aliases
{
"actions" : [
{
"add" : {
"index" : "test",
"alias" : "alias1",
"is_write_index" : true
}
}, {
"add" : {
"index" : "test2",
"alias" : "alias1",
"is_write_index" : false
}
}
]
}
--------------------------------------------------
// CONSOLE
// TEST[s/^/PUT test\nPUT test2\n/]

It is important to note that the order of these actions is important. The original write index
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is tricky - with our new model the order is not important but it is that you remove an explicit flag. I wonder if we should add a section over the "default" behavior and illustrate some examples.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

oh, woops. I thought I went through this. will update since it is left-over and currently wrong

must be unset, so that `test2` can be set as the write index.


[IMPORTANT]
=====================================
`is_write_index` will default to `true` for an index that only has one alias which did not explicitly specify a value.
=====================================

[float]
[[alias-adding]]
Expand Down Expand Up @@ -419,7 +507,8 @@ Response:
"term" : {
"year" : 2016
}
}
},
"is_write_index": false
}
}
}
Expand Down Expand Up @@ -448,7 +537,8 @@ Response:
"term" : {
"year" : 2016
}
}
},
"is_write_index": false
}
}
}
Expand Down Expand Up @@ -477,7 +567,8 @@ Response:
"term" : {
"year" : 2016
}
}
},
"is_write_index": false
}
}
}
Expand Down
2 changes: 1 addition & 1 deletion docs/reference/indices/rollover-index.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -141,7 +141,7 @@ GET _alias
{
"logs-2016.10.31-000002": {
"aliases": {
"logs_write": {}
"logs_write": { "is_write_index": true }
}
},
"logs-2016.10.31-1": {
Expand Down
5 changes: 4 additions & 1 deletion rest-api-spec/src/main/resources/rest-api-spec/test/indices.create/10_basic.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -74,18 +74,21 @@
filter:
term:
field : value
test_dlias:
is_write_index: true

- do:
indices.get_alias:
index: test_index

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias.search_routing: b}
- match: {test_index.aliases.test_blias.index_routing: b}
- is_false: test_index.aliases.test_blias.filter
- match: {test_index.aliases.test_clias.filter.term.field: value}
- is_false: test_index.aliases.test_clias.index_routing
- is_false: test_index.aliases.test_clias.search_routing
- is_true: test_index.aliases.test_dlias.is_write_index

---
"Create index with no type mappings":
Expand Down
82 changes: 41 additions & 41 deletions rest-api-spec/src/main/resources/rest-api-spec/test/indices.get_alias/10_basic.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -27,10 +27,10 @@ setup:
- do:
indices.get_alias: {}

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_blias: { is_write_index: false }}
- match: {test_index_3.aliases: {}}

---
Expand All @@ -44,10 +44,10 @@ setup:
indices.get_alias:
name: _all

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_blias: { is_write_index: false }}
- is_false: test_index_3

---
Expand All @@ -61,10 +61,10 @@ setup:
indices.get_alias:
name: _all

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_blias: { is_write_index: false }}
- is_false: test_index_3

---
Expand All @@ -74,8 +74,8 @@ setup:
indices.get_alias:
index: test_index

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- is_false: test_index_2

---
Expand All @@ -88,10 +88,10 @@ setup:
indices.get_alias:
index: _all

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_blias: { is_write_index: false }}
- match: {myindex.aliases: {}}

---
Expand All @@ -104,10 +104,10 @@ setup:
indices.get_alias:
index: "*"

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_blias: { is_write_index: false }}
- match: {myindex.aliases: {}}

---
Expand All @@ -130,7 +130,7 @@ setup:
index: test_index
name: test_alias

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- is_false: test_index.aliases.test_blias
- is_false: test_index_2

Expand All @@ -142,8 +142,8 @@ setup:
index: test_index
name: _all

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- is_false: test_index_2

---
Expand All @@ -154,8 +154,8 @@ setup:
index: test_index
name: '*'

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- is_false: test_index_2

---
Expand All @@ -165,7 +165,7 @@ setup:
index: test_index
name: 'test_a*'

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- is_false: test_index.aliases.test_blias
- is_false: test_index_2

Expand All @@ -177,8 +177,8 @@ setup:
index: test_index
name: 'test_alias,test_blias'

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_blias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index.aliases.test_blias: { is_write_index: false }}
- is_false: test_index_2

---
Expand All @@ -188,8 +188,8 @@ setup:
indices.get_alias:
name: test_alias

- match: {test_index.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- is_false: test_index.aliases.test_blias
- is_false: test_index_2.aliases.test_blias

Expand All @@ -201,8 +201,8 @@ setup:
index: _all
name: test_alias

- match: {test_index.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- is_false: test_index.aliases.test_blias
- is_false: test_index_2.aliases.test_blias

Expand All @@ -214,8 +214,8 @@ setup:
index: '*'
name: test_alias

- match: {test_index.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- is_false: test_index.aliases.test_blias
- is_false: test_index_2.aliases.test_blias

Expand All @@ -227,7 +227,7 @@ setup:
index: '*2'
name: test_alias

- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- is_false: test_index.aliases.test_alias
- is_false: test_index.aliases.test_blias
- is_false: test_index_2.aliases.test_blias
Expand All @@ -240,8 +240,8 @@ setup:
index: test_index,test_index_2
name: test_alias

- match: {test_index.aliases.test_alias: {}}
- match: {test_index_2.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: false }}
- match: {test_index_2.aliases.test_alias: { is_write_index: false }}
- is_false: test_index.aliases.test_blias
- is_false: test_index_2.aliases.test_blias

Expand All @@ -267,7 +267,7 @@ setup:

- match: { 'status': 404 }
- match: { 'error': 'alias [non-existent] missing' }
- match: { test_index.aliases.test_alias: { } }
- match: { test_index.aliases.test_alias: { is_write_index: false } }

---
"Existent and non-existent aliases returns 404 and the existing alias":
Expand All @@ -279,7 +279,7 @@ setup:

- match: { 'status': 404 }
- match: { 'error': 'aliases [another-non-existent,non-existent] missing' }
- match: { test_index.aliases.test_alias: { } }
- match: { test_index.aliases.test_alias: { is_write_index: false } }

---
"Getting alias on an non-existent index should return 404":
Expand Down
2 changes: 1 addition & 1 deletion rest-api-spec/src/main/resources/rest-api-spec/test/indices.put_alias/10_basic.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
indices.get_alias:
name: test_alias

- match: {test_index.aliases.test_alias: {}}
- match: {test_index.aliases.test_alias: { is_write_index: true }}

---
"Can't create alias with invalid characters":
Expand Down
Loading