Feat: enable dynamic profile field mapping via env variables for oidc…

… response (#1041)

* feat: Enable dynamic profile field mapping via env variables

* add oidc groups to access groups

* fix: minor fix for typo

* fix: set default oidc displayName to userInfo.name

* Added removal of duplicates post-array concatenation

* fixed typo and wrong user profile schema

* Allow for override of configuration

As not all config settings can be made as environment variables,
provide a override mechanism to allow (build-time) configuration
adjustments.

An example for the  graphql access groups provider will be in the next
commit.

Change-Id: I8dc82ca4f0ac0a1b60fa47eadb147c228a77b841

* Make the acccess groups service configurable

Instead of requiring an explict service provider for each facility,
use extended configurations.

Basic enabling/disabling is implemented in the standard config via
environment vars, GraphQL needs extendend configuration via
localconfiguration.

Change-Id: I2ed630bac8f1f66d4f754e5b95d6b232ec63cf3d

* Add example graphql handler

Change-Id: I832b6d9e2680aa8423441924de59c4157b50c8e6

* removed redundant code

* OIDC userinfo and user query settings have been made configurable

* improved if condition for parseQueryFilter

* fixed wrong default logger method integration

* improved logger message format

* get accessGroupsProperty from userPayload for AccesGroupFromPayloadService

* fix access-group-from-payload unit test fail

* refactor: improved readability of parseQueryFilter in oidc.strategy file

* fixed defaultLogger to log message without undefined even second parameter is not given.

* moved externalId and provider of create-user-identity dto to update-user-identity dto

* minor refactoring

* README updates for new environment variables

* Update README.md

---------

Co-authored-by: Björn Pedersen <[email protected]>
Co-authored-by: Max Novelli <[email protected]>

README.md

@@ -89,9 +89,15 @@ Valid environment variables for the .env file. See [.env.example](/.env.example)
 - `DATASET_CREATION_VALIDATION_REGEX` [string] Regular expression validation for new dataset request. Default value: ""
 - `PROPOSAL_GROUPS` [string] _Optional_ Comma separated list of proposal groups with permission to create any proposals. Example: "proposaladmin, proposalingestor". For more details check: [Scicat Documentation](https://scicatproject.github.io/documentation/Development/v4.x/backend/authorization.html)
 - `SAMPLE_GROUPS` [string] _Optional_ Comma separated list of sample groups with permission to create any samples. Example: "sampleadmin, sampleingestor". For more details check: [Scicat Documentation](https://scicatproject.github.io/documentation/Development/v4.x/backend/authorization.html)
-- `ACCESS_GROUPS_STATIC_VALUES` [string] _Optional_ Comma separated list of access groups automatically assigned to all users. Example: "scicat, user"
+
+- `ACCESS_GROUPS_GRAPHQL_ENABLED` [string] _Optional_ Flag to enable/disable the graphql service get access groups. In order to use this service following variables needs to be configured: `ACCESS_GROUP_SERVICE_TOKEN`, `ACCESS_GROUP_SERVICE_API_URL` and `ACCESS_GROUP_SERVICE_HANDLER` respectively. Values true or false. Defaults to true.
 - `ACCESS_GROUPS_SERVICE_TOKEN` [string] _Optional_ Authentication token used if access groups are obtained from a third party service. This value is not used by the vanilla installation, but only if the instance is customized to use an external service to provide user groups, like the ESS example
 - `ACCESS_GROUP_SERVICE_API_URL` [string] _Optional_ URL of the service providing the users' access groups. This value is not used by the vanilla installation, but only if the instance is customized to use an external service to provide user groups, like the ESS example
+- `ACCESS_GROUP_SERVICE_HANDLER` [string] _Optional_ Configuration property that points to the source of a module. This module provides a specific responseProcessor function for handling GraphQL responses and a query template for making GraphQL requests
+- `ACCESS_GROUPS_STATIC_ENABLED` [string] _Optional_ Flag to enable/disable automatic assignment of predefined access groups to all users. Values true or false. Defaults to true.
+- `ACCESS_GROUPS_STATIC_VALUES` [string] _Optional_ Comma separated list of access groups automatically assigned to all users. Example: "scicat, user"
+- `ACCESS_GROUPS_OIDCPAYLOAD_ENABLED` [string] _Optional_ Flag to enable/disable fetching access groups directly from OIDC response. Requires specifying a field via `OIDC_ACCESS_GROUPS_PROPERTY` to extract access groups. Defaults to false
+
 - `DOI_PREFIX` [string] The facility DOI prefix, with trailing slash.
 - `EXPRESS_SESSION_SECRET` [string] _Optional_ Secret used to set up express session.
 - `HTTP_MAX_REDIRECTS` [number] _Optional_ Max redirects for http requests. Defaults to 5.
@@ -110,6 +116,18 @@ Valid environment variables for the .env file. See [.env.example](/.env.example)
 - `OIDC_SCOPE` [string] _Optional_ Space separated list of the info returned by the oidc service. Example: "openid profile email"
 - `OIDC_SUCCESS_URL` [string] _Optional_ SciCat Frontend auth-callback URL. Required in order to pass user credentials to SciCat Frontend after OIDC login. Example: https://myscicatfrontend/auth-callback
 - `OIDC_ACCESS_GROUPS` [string] _Optional_ Functionality is still unclear.
+- `OIDC_ACCESS_GROUPS_PROPERTY` [string] _Optional_ Target field to get the access groups value from OIDC response.
+- `OIDC_USERINFO_MAPPING_FIELD_USERNAME` [string] _Optional_ comma-separated list. Specifies the fields from the OIDC response to concatenate and use as the user's profile username. For example, setting `OIDC_USERINFO_MAPPING_FIELD_USERNAME="iss, sub"` combines the iss (issuer) and sub (subject) values from the OIDC response, resulting in a username like `myIssuer_myUserName`. This allows for customizable username definitions based on OIDC response attributes. Defaults to "preferred_username" || "name"
+- `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME` [string] _Optional_ Specifies the fields from the OIDC response and use as the user's profile displayname. For example, setting `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME="preferred_username"` use displayName value from the OIDC response, resulting in a displayname like `myPreferredName`. This allows for customizable displayname definitions based on OIDC response attributes. Defaults to "name"
+- `OIDC_USERINFO_MAPPING_FIELD_EMAIL` [string] _Optional_ Same as `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME`. Defaults to "email"
+- `OIDC_USERINFO_MAPPING_FIELD_FAMILYNAME` [string] _Optional_ Same as `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME`. Defaults to "family_name"
+- `OIDC_USERINFO_MAPPING_FIELD_ID` [string] _Optional_ Same as `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME`. Defaults to "sub" || "user_id"
+- `OIDC_USERINFO_MAPPING_FIELD_THUMBNAILPHOTO` [string] _Optional_ Same as `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME`. Defaults to "thumbnailPhoto"
+- `OIDC_USERINFO_MAPPING_FIELD_PROVIDER` [string] _Optional_ Same as `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME`. Defaults to "iss"
+- `OIDC_USERINFO_MAPPING_FIELD_GROUP` [string] _Optional_ Same as `OIDC_USERINFO_MAPPING_FIELD_DISPLAYNAME`. Defaults to "groups"
+- `OIDC_USERQUERY_OPERATOR` [string] _Optional_ Specifies the operator ("or" or "and") for UserModel.findOne queries, determining the logic used to match fields like "username" or "email". Example: `UserModel.findOne({$or: {"username":"testUser", "email":"[email protected]"}})`. Defaults to "or"
+- `OIDC_USERQUERY_FILTER` [string] _Optional_ Defines key-value pairs for UserModel.findOne queries, using a "key:value" format. Values should correspond to fields in the userProfile object. For instance,` OIDC_USERQUERY_FILTER="username:sub, email:email"` maps to `userProfile.sub` and `userProfile.email` respectively. Defaults to "username:username, email:email"
+
 - `LOGBOOK_ENABLED` [string] _Optional_ Flag to enable/disable the Logbook endpoints. Values "yes" or "no". Defaults to "no".
 - `LOGBOOK_BASE_URL` [string] _Optional_ The base URL to the Logbook API. Only required if Logbook is enabled.
 

src/auth/access-group-provider/access-group-from-graphql-api-call.service.ts

@@ -5,7 +5,6 @@ import { Injectable, Logger } from "@nestjs/common";
 import { UserPayload } from "../interfaces/userPayload.interface";
 import { HttpService } from "@nestjs/axios";
 import { firstValueFrom } from "rxjs";
-
 /**
  * This service is used to fetch access groups from a GraphQL API.
  */
@@ -27,14 +26,19 @@ export class AccessGroupFromGraphQLApiService extends AccessGroupService {
   ): Promise<string[]> {
     const userId = userPayload.userId as string;
     const query = this.graphqlTemplateQuery.replace("{{userId}}", userId);
-    const response = await this.callGraphQLApi(query);
-    const accessGroups = this.responseProcessor(response);
+    try {
+      const response = await this.callGraphQLApi(query);
+      const accessGroups = this.responseProcessor(response);
 
-    Logger.log(
-      "Access groups from graphql api call getAccessGroups: " +
-        accessGroups.join(","),
-    );
-    return accessGroups;
+      Logger.log(
+        accessGroups,
+        "AccessGroupFromGraphQLApiService getAccessGroups:",
+      );
+      return accessGroups;
+    } catch (error) {
+      Logger.error(error, "AccessGroupFromGraphQLApiService");
+      return [];
+    }
   }
 
   async callGraphQLApi(query: string): Promise<Record<string, unknown>> {

src/auth/access-group-provider/access-group-from-multiple-providers.service.ts

@@ -1,4 +1,4 @@
-import { Injectable } from "@nestjs/common";
+import { Injectable, Logger } from "@nestjs/common";
 import { UserPayload } from "../interfaces/userPayload.interface";
 import { AccessGroupService } from "./access-group.service";
 

src/auth/access-group-provider/access-group-from-payload.service.spec.ts

@@ -7,9 +7,7 @@ describe("AccessGroupFromPayloadService", () => {
   let service: AccessGroupFromPayloadService;
 
   const mockConfigService = {
-    get: () => ({
-      accessGroups: "accessGroups",
-    }),
+    get: () => "access_group_property",
   };
 
   beforeEach(async () => {
@@ -30,14 +28,15 @@ describe("AccessGroupFromPayloadService", () => {
   });
 
   it("Should resolve access groups", async () => {
-    const expected = ["AAA", "BBB"];
-    const actual = await service.getAccessGroups({
+    const userPayload = {
       userId: "test_user",
-      accessGroupProperty: "access_group_property",
+      accessGroupProperty: "testGroups",
       payload: {
-        access_group_property: expected,
+        testGroups: ["AAA", "BBB"],
       },
-    } as UserPayload);
+    };
+    const expected = userPayload.payload.testGroups;
+    const actual = await service.getAccessGroups(userPayload as UserPayload);
     expect(actual).toEqual(expected);
   });
 });

src/auth/access-group-provider/access-group-from-payload.service.ts

@@ -17,8 +17,7 @@ export class AccessGroupFromPayloadService extends AccessGroupService {
     //const defaultAccessGroups: string[] = [];
     let accessGroups: string[] = [];
 
-    const accessGroupsProperty = userPayload?.accessGroupProperty;
-
+    const accessGroupsProperty = userPayload.accessGroupProperty;
     if (accessGroupsProperty) {
       const payload: Record<string, unknown> | undefined = userPayload.payload;
       if (
@@ -30,11 +29,9 @@ export class AccessGroupFromPayloadService extends AccessGroupService {
             ? (payload[accessGroupsProperty] as string[])
             : [];
       }
+      Logger.log(accessGroups, "AccessGroupFromPayloadService");
     }
 
-    Logger.log(
-      "Access groups AccessGroupFromPayloadService : " + accessGroups.join(","),
-    );
     return accessGroups;
   }
 }

src/auth/access-group-provider/access-group-service-factory.ts

@@ -1,17 +1,79 @@
 import { ConfigService } from "@nestjs/config";
 import { AccessGroupFromStaticValuesService } from "./access-group-from-static-values.service";
 import { AccessGroupService } from "./access-group.service";
-
+import { AccessGroupFromGraphQLApiService } from "./access-group-from-graphql-api-call.service";
+import { AccessGroupFromPayloadService } from "./access-group-from-payload.service";
+import { HttpService } from "@nestjs/axios";
+import { AccessGroupFromMultipleProvidersService } from "./access-group-from-multiple-providers.service";
+import { Logger } from "@nestjs/common";
+import { ConfigModule } from "@nestjs/config";
 /*
  * this is the default function which provides an empty array as groups
  */
 export const accessGroupServiceFactory = {
+  imports: [ConfigModule],
   provide: AccessGroupService,
   useFactory: (configService: ConfigService) => {
-    const accessGroupsStaticValues = configService.get(
-      "accessGroupsStaticValues",
+    Logger.debug("Service factory starting", "accessGroupServiceFactory");
+    const accessGroupsStaticConfig = configService.get(
+      "accessGroupsStaticConfig",
+    );
+    const accessGroupsGraphQlConfig = configService.get(
+      "accessGroupsGraphQlConfig",
     );
-    return new AccessGroupFromStaticValuesService(accessGroupsStaticValues);
+    const accessGroupsOIDCPayloadConfig = configService.get(
+      "accessGroupsOIDCPayloadConfig",
+    );
+
+    const accessGroupServices: AccessGroupService[] = [];
+    if (accessGroupsStaticConfig?.enabled == true) {
+      Logger.log(
+        JSON.stringify(accessGroupsStaticConfig),
+        "loading static processor",
+      );
+      accessGroupServices.push(
+        new AccessGroupFromStaticValuesService(accessGroupsStaticConfig.value),
+      );
+    }
+    if (accessGroupsOIDCPayloadConfig?.enabled == true) {
+      Logger.log(
+        JSON.stringify(accessGroupsOIDCPayloadConfig),
+        "loading oidc processor",
+      );
+      accessGroupServices.push(
+        new AccessGroupFromPayloadService(configService),
+      );
+    }
+
+    if (accessGroupsGraphQlConfig?.enabled == true) {
+      Logger.log(
+        JSON.stringify(accessGroupsGraphQlConfig),
+        "loading graphql processor",
+      );
+
+      import(accessGroupsGraphQlConfig.responseProcessorSrc).then(
+        (rpModule) => {
+          const gh = rpModule.graphHandler;
+          const responseProcessor: (
+            response: Record<string, unknown>,
+          ) => string[] = gh.responseProcessor;
+          const graphqlTemplateQuery: string = gh.graphqlTemplateQuery;
+          accessGroupServices.push(
+            new AccessGroupFromGraphQLApiService(
+              graphqlTemplateQuery,
+              accessGroupsGraphQlConfig.apiUrl,
+              {
+                Authorization: `Bearer ${accessGroupsGraphQlConfig.token}`,
+              },
+              responseProcessor,
+              new HttpService(),
+            ),
+          );
+        },
+      );
+    }
+
+    return new AccessGroupFromMultipleProvidersService(accessGroupServices);
   },
   inject: [ConfigService],
 };

src/auth/interfaces/oidc-user.interface.ts

@@ -0,0 +1,16 @@
+export interface IOidcUserInfoMapping {
+  id: string;
+  username: string;
+  displayName: string;
+  familyName: string;
+  email: string;
+  thumbnailPhoto: string;
+  groups?: string[];
+  provider?: string;
+  [key: string]: string | string[] | undefined;
+}
+
+export interface IOidcUserQueryMapping {
+  operator: string;
+  filter: string[];
+}