Add variable Extension support (#1328)

* Added root level and item level variable extension support in Questionnaire * Refactored QuestionnaireViewModel to make code more readable * Refactored ancestors logic and add a Path to Variables map to save and retrieve Variables * Refactored variables code to make it more readable * Refactored ancestors variables code to make it readable * Refactored to make the variables update for all siblings and ancestors onAnswerChanges * Code readability improvements * Refactored, removed the extra functions * Move Extension URl to MoreQuestionnaireItemComponents file * Added unit tests for variable extension in QuestionnaireViewModelTest * Add more unit tests for variable extension * Added more unit tests for variable extension for invalid expresssions * Added some documemtation for Variables extensions * Fix spellings * Added unit tests for FHIRPathEnginehostServices * Fix unit test in FHIRPathEngineHostServices * Fixed wrong ArgumentMatchers on FHIRPathEngineHostServices file * Update Variables logic based on PR feedback * Update nullability of updateVariable in quetionnaireViewModel * Docs updated * Docs updated * Added more tests to FHIRPathEngineHostServices * Added more unit test for QuestionnaireViewModel * Update evaluateVariable method to make it re-usable * updated variableName * Removed unnecesary configs and Test runner * Updated code to add dependent variables logic in pathToVariablesMap * Add findDependents logic to find linkid and variable dependents * Fix spotless check * Fix broken unit tests * Fix broken unit tests, make variable null instead of empty * Add more unit tests for QuestionnaireViewModel * Add more unit tests * Fix regex for linkId and variable name * Added more tests * Added documentation * Refactor some unit tests of QuestionnaireViewModel * Added more test coverage * Added more test case and require check for expression language * Fix regex to match variable name * Added missing unit tests for page flow * Fix spotless Check * Fix spotless check * Spotless apply * Changed solution approach, wrote a method to use it to calculate expression on the fly * Remove questionnairePreOrderList * Refactored findVariable method to make it more readable * Removed broken, unrelated unit tests of QuestionnaireViewModel * Make evaluateExpression internal to access it for use and testing * Spotless apply * Added unit tests for root level and questionnaire item, origin variables * Added more tests for variables in QuestionnaireViewModelTest * Added failure cases and their test cases for fhirpath expressions * Remove un-related unit tests * feedback resolved around documentation * Fix some documentation and spotless apply * Revert "Fix some documentation and spotless apply" This reverts commit 9aafd1b. * Refactored documentation * spelling correction * Resolved feedback comments around documentation and immutability of map in catalog/QuestionnaireViewModel * Fix docs * Fix docs , apply spotless * Fix docs , apply spotless * Separate out apis for varaibles expression into ExpressionEvaluator * Changed Junit Assertions to Truth Assertions * Add a unit test, update documentation, spotless apply * Rename unit test * Added more documentation * Improve code readibility in Expression Evaluator and remove commented code * Fix documentation * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Added docs, Did refatoring in ExpressionEvaluator * Fix documentation * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Refactored methods to make it more readable * Update datacapture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt Co-authored-by: Jing Tang <[email protected]> * Renamed maps and fix documentation * Fix documentation * Renamed variablesMap * Added a todo, apply spotless Co-authored-by: maimoonak <[email protected]> Co-authored-by: Jing Tang <[email protected]>
google · Sep 1, 2022 · 9e43cce · 9e43cce
1 parent b12ff3a
commit 9e43cce
Show file tree

Hide file tree

Showing 8 changed files with 1,008 additions and 3 deletions.
diff --git a/...ture/src/main/java/com/google/android/fhir/datacapture/MoreQuestionnaireItemComponents.kt b/...ture/src/main/java/com/google/android/fhir/datacapture/MoreQuestionnaireItemComponents.kt
@@ -59,9 +59,29 @@ internal const val EXTENSION_HIDDEN_URL =
 internal const val EXTENSION_ENTRY_FORMAT_URL =
   "http://hl7.org/fhir/StructureDefinition/entryFormat"
 
-internal const val ITEM_ENABLE_WHEN_EXPRESSION_URL: String =
+internal const val EXTENSION_ENABLE_WHEN_EXPRESSION_URL: String =
   "http://hl7.org/fhir/uv/sdc/StructureDefinition/sdc-questionnaire-enableWhenExpression"
 
+internal const val EXTENSION_VARIABLE_URL = "http://hl7.org/fhir/StructureDefinition/variable"
+
+internal val Questionnaire.QuestionnaireItemComponent.variableExpressions: List<Expression>
+  get() =
+    this.extension.filter { it.url == EXTENSION_VARIABLE_URL }.map { it.castToExpression(it.value) }
+
+/**
+ * Finds the specific variable name [String] at the questionnaire item
+ * [Questionnaire.QuestionnaireItemComponent]
+ *
+ * @param variableName the [String] to match the variable
+ *
+ * @return an [Expression]
+ */
+internal fun Questionnaire.QuestionnaireItemComponent.findVariableExpression(
+  variableName: String
+): Expression? {
+  return variableExpressions.find { it.name == variableName }
+}
+
 // Item control code, or null
 internal val Questionnaire.QuestionnaireItemComponent.itemControl: ItemControlTypes?
   get() {
@@ -225,7 +245,7 @@ fun Questionnaire.QuestionnaireItemComponent.createQuestionnaireResponseItem():
 // Return expression if QuestionnaireItemComponent has ENABLE WHEN EXPRESSION URL
 val Questionnaire.QuestionnaireItemComponent.enableWhenExpression: Expression?
   get() {
-    return this.extension.firstOrNull { it.url == ITEM_ENABLE_WHEN_EXPRESSION_URL }?.let {
+    return this.extension.firstOrNull { it.url == EXTENSION_ENABLE_WHEN_EXPRESSION_URL }?.let {
       it.value as Expression
     }
   }

diff --git a/datacapture/src/main/java/com/google/android/fhir/datacapture/MoreQuestionnaires.kt b/datacapture/src/main/java/com/google/android/fhir/datacapture/MoreQuestionnaires.kt
@@ -17,6 +17,7 @@
 package com.google.android.fhir.datacapture
 
 import org.hl7.fhir.r4.model.CanonicalType
+import org.hl7.fhir.r4.model.Expression
 import org.hl7.fhir.r4.model.Questionnaire
 
 /**
@@ -31,6 +32,19 @@ val Questionnaire.targetStructureMap: String?
     return if (extensionValue is CanonicalType) extensionValue.valueAsString else null
   }
 
+internal val Questionnaire.variableExpressions: List<Expression>
+  get() =
+    this.extension.filter { it.url == EXTENSION_VARIABLE_URL }.map { it.castToExpression(it.value) }
+
+/**
+ * Finds the specific variable name [String] at questionnaire [Questionnaire] level
+ *
+ * @param variableName the [String] to match the variable at questionnaire [Questionnaire] level
+ * @return [Expression] the matching expression
+ */
+internal fun Questionnaire.findVariableExpression(variableName: String): Expression? =
+  variableExpressions.find { it.name == variableName }
+
 /**
  * See
  * [Extension: target structure map](http://build.fhir.org/ig/HL7/sdc/StructureDefinition-sdc-questionnaire-targetStructureMap.html)

diff --git a/datacapture/src/main/java/com/google/android/fhir/datacapture/QuestionnaireViewModel.kt b/datacapture/src/main/java/com/google/android/fhir/datacapture/QuestionnaireViewModel.kt
@@ -155,6 +155,30 @@ internal class QuestionnaireViewModel(application: Application, state: SavedStat
     }
   }
 
+  /** The map from each item in the [Questionnaire] to its parent. */
+  private var questionnaireItemParentMap:
+    Map<Questionnaire.QuestionnaireItemComponent, Questionnaire.QuestionnaireItemComponent>
+
+  init {
+    /** Adds each child-parent pair in the [Questionnaire] to the parent map. */
+    fun buildParentList(
+      item: Questionnaire.QuestionnaireItemComponent,
+      questionnaireItemToParentMap: ItemToParentMap
+    ) {
+      for (child in item.item) {
+        questionnaireItemToParentMap[child] = item
+        buildParentList(child, questionnaireItemToParentMap)
+      }
+    }
+
+    questionnaireItemParentMap =
+      buildMap {
+        for (item in questionnaire.item) {
+          buildParentList(item, this)
+        }
+      }
+  }
+
   /** The map from each item in the [QuestionnaireResponse] to its parent. */
   private val questionnaireResponseItemParentMap =
     mutableMapOf<
@@ -651,6 +675,9 @@ internal class QuestionnaireViewModel(application: Application, state: SavedStat
   }
 }
 
+typealias ItemToParentMap =
+  MutableMap<Questionnaire.QuestionnaireItemComponent, Questionnaire.QuestionnaireItemComponent>
+
 /** Questionnaire state for the Fragment to consume. */
 internal data class QuestionnaireState(
   /** The items that should be currently-rendered into the Fragment. */

diff --git a/...capture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt b/...capture/src/main/java/com/google/android/fhir/datacapture/fhirpath/ExpressionEvaluator.kt
@@ -0,0 +1,285 @@
+/*
+ * Copyright 2022 Google LLC
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ *       http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package com.google.android.fhir.datacapture.fhirpath
+
+import ca.uhn.fhir.context.FhirContext
+import ca.uhn.fhir.context.FhirVersionEnum
+import com.google.android.fhir.datacapture.findVariableExpression
+import com.google.android.fhir.datacapture.variableExpressions
+import org.hl7.fhir.exceptions.FHIRException
+import org.hl7.fhir.r4.hapi.ctx.HapiWorkerContext
+import org.hl7.fhir.r4.model.Base
+import org.hl7.fhir.r4.model.Expression
+import org.hl7.fhir.r4.model.Questionnaire
+import org.hl7.fhir.r4.model.QuestionnaireResponse
+import org.hl7.fhir.r4.utils.FHIRPathEngine
+import timber.log.Timber
+
+/**
+ * Evaluates an expression and returns its result.
+ *
+ * Expressions can be defined at questionnaire level and questionnaire item level. This
+ * [ExpressionEvaluator] supports evaluation of
+ * [variable expression](http://hl7.org/fhir/R4/extension-variable.html) defined at either
+ * questionnaire level or questionnaire item level.
+ *
+ * TODO(https://github.com/google/android-fhir/issues/1575): Add a global map to apply variable
+ * values if already evaluated to avoid re-calculation
+ */
+object ExpressionEvaluator {
+
+  private val reservedVariables =
+    listOf("sct", "loinc", "ucum", "resource", "rootResource", "context", "map-codes")
+
+  /**
+   * Finds all the matching occurrences of variables. For example, when we apply regex to the
+   * expression "%X + %Y", if we simply groupValues, it returns [%X, X], [%Y, Y] The group with
+   * index 0 is always the entire matched string (%X and %Y). The indices greater than 0 represent
+   * groups in the regular expression (X and Y) so we groupValues by first index to get only the
+   * variables name without % as prefix i.e, ([X, Y])
+   *
+   * If we apply regex to the expression "X + Y", it returns nothing as there are no matching groups
+   * in this expression
+   */
+  private val variableRegex = Regex("[%]([A-Za-z0-9\\-]{1,64})")
+
+  private val fhirPathEngine: FHIRPathEngine =
+    with(FhirContext.forCached(FhirVersionEnum.R4)) {
+      FHIRPathEngine(HapiWorkerContext(this, this.validationSupport)).apply {
+        hostServices = FHIRPathEngineHostServices
+      }
+    }
+
+  /**
+   * Evaluates variable expression defined at questionnaire item level and returns the evaluated
+   * result.
+   *
+   * Parses the expression using regex [Regex] for variable (For example: A variable name could be
+   * %weight) and build a list of variables that the expression contains and for every variable, we
+   * first find it at questionnaire item, then up in the ancestors and then at questionnaire level,
+   * if found we get their expressions and pass them into the same function to evaluate its value
+   * recursively, we put the variable name and its evaluated value into the map [Map] to use this
+   * map to pass into fhirPathEngine's evaluate method to apply the evaluated values to the
+   * expression being evaluated.
+   *
+   * @param expression the [Expression] Variable expression
+   * @param questionnaire the [Questionnaire] respective questionnaire
+   * @param questionnaireResponse the [QuestionnaireResponse] respective questionnaire response
+   * @param questionnaireItemParentMap the [Map<Questionnaire.QuestionnaireItemComponent,
+   * Questionnaire.QuestionnaireItemComponent>] of child to parent
+   * @param questionnaireItem the [Questionnaire.QuestionnaireItemComponent] where this expression
+   * is defined,
+   *
+   * @return [Base] the result of expression
+   */
+  internal fun evaluateQuestionnaireItemVariableExpression(
+    expression: Expression,
+    questionnaire: Questionnaire,
+    questionnaireResponse: QuestionnaireResponse,
+    questionnaireItemParentMap:
+      Map<Questionnaire.QuestionnaireItemComponent, Questionnaire.QuestionnaireItemComponent>,
+    questionnaireItem: Questionnaire.QuestionnaireItemComponent
+  ): Base? {
+    require(
+      questionnaireItem.variableExpressions.any {
+        it.name == expression.name && it.expression == expression.expression
+      }
+    ) { "The expression should come from the same questionnaire item" }
+
+    val dependentVariablesMap = buildMap {
+      findDependentVariables(expression).forEach { variableName ->
+        findAndEvaluateVariable(
+          variableName,
+          questionnaireItem,
+          questionnaire,
+          questionnaireResponse,
+          questionnaireItemParentMap,
+          this
+        )
+      }
+    }
+    return evaluateVariable(expression, questionnaireResponse, dependentVariablesMap)
+  }
+
+  /**
+   * Evaluates variable expression defined at questionnaire level and returns the evaluated result.
+   *
+   * Parses the expression using [Regex] for variable (For example: A variable name could be
+   * %weight) and build a list of variables that the expression contains and for every variable, we
+   * first find it at questionnaire level, if found we get their expressions and pass them into the
+   * same function to evaluate its value recursively, we put the variable name and its evaluated
+   * value into the map [Map] to use this map to pass into fhirPathEngine's evaluate method to apply
+   * the evaluated values to the expression being evaluated.
+   *
+   * @param expression the [Expression] Variable expression
+   * @param questionnaire the [Questionnaire] respective questionnaire
+   * @param questionnaireResponse the [QuestionnaireResponse] respective questionnaire response
+   *
+   * @return [Base] the result of expression
+   */
+  internal fun evaluateQuestionnaireVariableExpression(
+    expression: Expression,
+    questionnaire: Questionnaire,
+    questionnaireResponse: QuestionnaireResponse
+  ): Base? {
+    val variableMap =
+      buildMap<String, Base?> {
+        findDependentVariables(expression).forEach { variableName ->
+          questionnaire.findVariableExpression(variableName)?.let { expression ->
+            put(
+              expression.name,
+              evaluateQuestionnaireVariableExpression(
+                expression,
+                questionnaire,
+                questionnaireResponse
+              )
+            )
+          }
+        }
+      }
+    return evaluateVariable(expression, questionnaireResponse, variableMap)
+  }
+
+  private fun findDependentVariables(expression: Expression) =
+    variableRegex.findAll(expression.expression).map { it.groupValues[1] }.toList().filterNot {
+      variable ->
+      reservedVariables.contains(variable)
+    }
+
+  /**
+   * Finds the dependent variables at questionnaire item level first, then in ancestors and then at
+   * questionnaire level
+   *
+   * @param variableName the [String] to match the variable in the ancestors
+   * @param questionnaireItem the [Questionnaire.QuestionnaireItemComponent] from where we have to
+   * track hierarchy up in the ancestors
+   * @param questionnaire the [Questionnaire] respective questionnaire
+   * @param questionnaireResponse the [QuestionnaireResponse] respective questionnaire response
+   * @param questionnaireItemParentMap the [Map<Questionnaire.QuestionnaireItemComponent,
+   * Questionnaire.QuestionnaireItemComponent>] of child to parent
+   * @param dependentVariablesMap the [Map<String, Base>] of dependent variables
+   */
+  private fun findAndEvaluateVariable(
+    variableName: String,
+    questionnaireItem: Questionnaire.QuestionnaireItemComponent,
+    questionnaire: Questionnaire,
+    questionnaireResponse: QuestionnaireResponse,
+    questionnaireItemParentMap:
+      Map<Questionnaire.QuestionnaireItemComponent, Questionnaire.QuestionnaireItemComponent>,
+    dependentVariablesMap: MutableMap<String, Base?>
+  ) {
+    when {
+      // First, check the questionnaire item itself
+      questionnaireItem.findVariableExpression(variableName) != null -> {
+        questionnaireItem.findVariableExpression(variableName)?.let { expression ->
+          dependentVariablesMap[expression.name] =
+            evaluateQuestionnaireItemVariableExpression(
+              expression,
+              questionnaire,
+              questionnaireResponse,
+              questionnaireItemParentMap,
+              questionnaireItem
+            )
+        }
+      }
+      // Secondly, check the ancestors of the questionnaire item
+      findVariableInAncestors(variableName, questionnaireItemParentMap, questionnaireItem) !=
+        null -> {
+        findVariableInAncestors(variableName, questionnaireItemParentMap, questionnaireItem)?.let {
+          (questionnaireItem, expression) ->
+          dependentVariablesMap[expression.name] =
+            evaluateQuestionnaireItemVariableExpression(
+              expression,
+              questionnaire,
+              questionnaireResponse,
+              questionnaireItemParentMap,
+              questionnaireItem
+            )
+        }
+      }
+      // Finally, check the variables defined on the questionnaire itself
+      else -> {
+        questionnaire.findVariableExpression(variableName)?.also { expression ->
+          dependentVariablesMap[expression.name] =
+            evaluateQuestionnaireVariableExpression(
+              expression,
+              questionnaire,
+              questionnaireResponse
+            )
+        }
+      }
+    }
+  }
+
+  /**
+   * Finds the questionnaire item having specific variable name [String] in the ancestors of
+   * questionnaire item [Questionnaire.QuestionnaireItemComponent]
+   *
+   * @param variableName the [String] to match the variable in the ancestors
+   * @param questionnaireItem the [Questionnaire.QuestionnaireItemComponent] whose ancestors we
+   * visit
+   * @param questionnaireItemParentMap the [Map<Questionnaire.QuestionnaireItemComponent,
+   * Questionnaire.QuestionnaireItemComponent>] of child to parent
+   * @return [Pair] containing [Questionnaire.QuestionnaireItemComponent] and [Expression]
+   */
+  private fun findVariableInAncestors(
+    variableName: String,
+    questionnaireItemParentMap:
+      Map<Questionnaire.QuestionnaireItemComponent, Questionnaire.QuestionnaireItemComponent>,
+    questionnaireItem: Questionnaire.QuestionnaireItemComponent
+  ): Pair<Questionnaire.QuestionnaireItemComponent, Expression>? {
+    var parent = questionnaireItemParentMap[questionnaireItem]
+    while (parent != null) {
+      val expression = parent.findVariableExpression(variableName)
+      if (expression != null) return Pair(parent, expression)
+
+      parent = questionnaireItemParentMap[parent]
+    }
+    return null
+  }
+
+  /**
+   * Evaluates the value of variable expression and return the evaluated value
+   *
+   * @param expression the [Expression] the expression to evaluate
+   * @param questionnaireResponse the [QuestionnaireResponse] respective questionnaire response
+   * @param dependentVariables the [Map] of Variable names to their values
+   *
+   * @return [Base] the result of expression
+   */
+  private fun evaluateVariable(
+    expression: Expression,
+    questionnaireResponse: QuestionnaireResponse,
+    dependentVariables: Map<String, Base?> = mapOf()
+  ) =
+    try {
+      require(expression.name?.isNotBlank() == true) {
+        "Expression name should be a valid expression name"
+      }
+
+      require(expression.hasLanguage() && expression.language == "text/fhirpath") {
+        "Unsupported expression language, language should be text/fhirpath"
+      }
+
+      fhirPathEngine
+        .evaluate(dependentVariables, questionnaireResponse, null, null, expression.expression)
+        .firstOrNull()
+    } catch (exception: FHIRException) {
+      Timber.w("Could not evaluate expression with FHIRPathEngine", exception)
+      null
+    }
+}