Extend judgments data model for extended usage in evaluation

[jzonthemtn]

Status quo how a judgment looks like:

{
  "_index": "judgments",
  "_id": "743ea9ae-41de-4b6a-80ee-fcdddaa712fc",
  "_score": 1,
  "_source": {
    "id": "743ea9ae-41de-4b6a-80ee-fcdddaa712fc",
    "queryId": "1",
    "query": "printer toner",
    "document": "B001C3CKRA",
    "judgment": 0,
    "judgmentsId": "f52a3831-5ca4-4127-9a39-97a923dd99fa",
    "timestamp": "2025-01-24T08:45:48.499Z",
    "judgmentAsMap": {
      "query_id": "1",
      "judgment": 0,
      "query": "printer toner",
      "document_id": "B001C3CKRA"
    }
  }
}

Judgments that were created as a result of one process all have the same judgmentsId.

Judgments can be uploaded by simply adding documents that follow the schema.

Extend the schema by the following attributes:

• judgment_set_type: either explicit or implicit, AI-generated judgments would count as explicit
• judgment_set_generator: describes who or what generated the judgment: a model (for example coec), a human, an AI system
• judgment_set_name: name of the judgment collection
• judgment_set_description: textual description of the judgments; optional
• judgment_set_parameters: For implicit judgments it's important to know what data it was created on; automatically filled by search eval app; two sub-fields start_date, end_date

Make the existing fields all follow the_underscore notation:

• queryId --> query_id
• judgmentsId --> judgment_set_id
• judgmentAsMap --> judgment_as_map //can be removed

Use cases enabled by the extension:

• Distinguish sets of judgments: which one do I want to use to run an evaluation job?
• In the future, use more than one set of judgments to do search quality evaluation runs. For example, combine two or more sets of explicit judgments to have a more complete picture. This would require a change in the search eval app as well.
• Add more information to a set of judgments: is this set judgments based on the behavior data of the last 30 or 90 days?