Adding to the documentation and tests for the _none pipeline

[masseyke]

The special pipeline name _none indicates that no ingest pipeline will run. This PR adds some documentation and tests to make it more clear that

1. If the request pipeline is _none then the default pipeline will not run
2. If the request pipeline is _none then the final pipeline (if it exists) will run

[github-actions]

Documentation preview:

• ✨ Changed pages

[elasticsearchmachine]

Pinging @elastic/es-docs (Team:Docs)

[elasticsearchmachine]

Pinging @elastic/es-data-management (Team:Data Management)

[mattc58]

@masseyke in your description you have both _none and none (no underscore). Is this a typo and are they both meant to be _none?

[masseyke]

@masseyke in your description you have both _none and none (no underscore). Is this a typo and are they both meant to be _none?

That was a typo.

[mattc58]

The only thing I'm iffy on here is the usability and hidden nature of this flag. It doesn't exactly scream skip_pipeline to me as a user. It's awesome that it's already there and we're finished, but...I kinda wonder if we should consider adding the parameter anyway as a synonym for this? I'm thinking about how it will look in the eventual Logstash pipeline config on the elasticsearch output block and it'll just look weird as pipeline => "_none". It's not the most self-documenting thing ever.

[masseyke]

The only thing I'm iffy on here is the usability and hidden nature of this flag. It doesn't exactly scream skip_pipeline to me as a user. It's awesome that it's already there and we're finished, but...I kinda wonder if we should consider adding the parameter anyway as a synonym for this? I'm thinking about how it will look in the eventual Logstash pipeline config on the elasticsearch output block and it'll just look weird as pipeline => "_none". It's not the most self-documenting thing ever.

It's a little awkward any way we do it. For example, saying pipeline=foo&skip_pipeline=true feels odd -- why even pass the pipeline?

[masseyke]

We could have _skip as a synonym for _none? So it would make more sense if you passed in pipeline => "_skip"?

[mattc58]

I think having two reserved words for the same thing might not be good either, hmm. I'd be interested in anyone else's thoughts. I guess since this feature is already here and (now has) tests and docs we could just roll with it.

[jbaiera]

Since this already exists, I don't think we should mirror the logic with another parameter; The same reason having two setting values that do the same thing being a little clunky applies.

My understanding is that the main use case for this setting in the first place is to skip running the default pipeline. The discussions we had about skipping the specific pipeline were more about handling an edge case that already made tenuous sense. Overall I think the existing _none value is a more elegant solution than our proposed changes, and on top of that, it already exists 😄

[jbaiera]

Suggested change

	(Optional, string) ID of the pipeline to use to preprocess incoming documents. The special pipeline
	name `_none` indicates no ingest pipeline will run, other than the final pipeline if it exists.
	(Optional, string) ID of the pipeline to use to preprocess incoming documents. If the index has a default ingest pipeline specified, then setting the value to `_none` disables the default ingest pipeline for this request. If a final pipeline is configured it will always run.

What do you think? Perhaps we could link to docs for both default pipelines and final pipelines?

[jbaiera]

https://www.elastic.co/guide/en/elasticsearch/reference/8.6/index-modules.html#index-default-pipeline

and

https://www.elastic.co/guide/en/elasticsearch/reference/8.6/index-modules.html#index-final-pipeline

Might also be a good idea to double check those sections. For some reason it states that _none will keep the final pipeline from running. Unless it means setting that on the index settings?

[masseyke]

Yeah that is saying that if you set the final pipeline to _none then it will keep the final pipeline from running. I'll make that a little more clear while I'm in here. Your wording sounds good to me -- I'll change it to something more like that.

[jbaiera]

LGTM!

[kilfoyle]

LGTM for the doc updates! 🚀