Allow all properties of an object to be queried and aggregated as top-level attributes

[felixbarny]

In order to properly map events that come in via the OpenTelemetry Protocol (OTLP), we'd like to have the ability to store attributes in an attributes and resource_attributes object.

For example:

{
  "resource_attributes": {
    "host.name": "foo"
  }
}

We still want to be able do queries and aggregations directly on host.name. While we could just store attributes at the top level, we'd be losing information on whether something is a resource attribute vs attribute and it would be difficult to convert that document back into OTLP without loss of information.

Maybe we can build on the foundations of the alias field type. However, it would need to be a generic kind of alias that makes attributes.* and resource_attributes.* available at the top level. Ideally, this should not have an impact on the field limit of an index.

We could, for example, create a special object field type (maybe root_object) or have a mapping parameter for object field types that optionally makes the fields within a document available at the top-level. Any defined subfield is also available as an alias at the same level that the object field is defined. These fields behave in queries and aggregations as if they had been defined at the root level. When returning fields from a search, they should probably only be returned with the prefix, for example ["attributes.host.name": ["my-host"]], rather than duplicated, such as ["attributes.host.name": ["my-host"], "host.name": ["my-host"]]

When there are multiple root_object definitions, there needs to be a declared order of precedence in which attributes are resolved. For example, if both attributes and resource_attributes define a service.name field, the one in resource_attributes should win.

This can also help with #98384 as we can add dynamic templates that define attributes.* and resource_attributes.* as time_series_dimension. Alternatively, this new field type can be marked as a time_series_dimension which implies that all its sub-fields are also dimensions.

Example mapping:

{
  "dynamic": "strict",
  "properties": {
    "resource_attributes": {
      "type": "root_object",
      "dynamic": true,
      "subobjects": false,
      "time_series_dimension": true
      "properties": {
        "host.name": {
          "type": "keyword"
        }
      }
    }
  }
}

cc @elastic/opentelemetry-leads @elastic/es-analytics-geo

[elasticsearchmachine]

Pinging @elastic/es-search (Team:Search)