[Extensions] Create a way to identify if an Extension REST Handler will create a scheduled job

[cwperks]

Some plugin actions, such as Anomaly Detector's Create Detector, create job definitions in an index owned by the plugin that Job Scheduler is aware of. The job definition contains a schedule section which specifies if the job runs on a CRON schedule or an an interval. This is how plugins make Job Scheduler aware that jobs exist and when to trigger the jobs to be run.

With extensions, the indices are still owned by the extension and the extension makes Job Scheduler aware of the index to look for new jobs. As part of handling async operations for extensions, the jobs definition also needs to store a token representative of the user who created the job so that Job Scheduler can request to issue new access tokens on behalf of the user.

These tokens are refresh_tokens and are tokens that themselves do not provide access to the cluster, however, Job Scheduler can request a new access token from the Security plugin on job invocation by utilizing this access token.

For this issue, a method needs to be devised to identify if an Extension REST Handler will create a scheduled job. For requests that create a scheduled job, a refresh_token (in addition to the access token) will be provided to the extension to store with the job definition.

[stephen-crawford]

[Triage] This is part of the Extensions project.

[davidlago]

@cwperks it seems like this might need to be closed in favor of different work, could you please update? Thanks!

[cwperks]

I will update this issue to reflect updates discovered since this was created. At some point the user details need to be saved on job creation. I believe its possible to hook into here within JobScheduler: https://github.com/opensearch-project/job-scheduler/blob/main/src/main/java/org/opensearch/jobscheduler/sweeper/JobSweeper.java#L190-L209

I envision that JobScheduler will be able to get the IdentityService through dependency injection and there may be a method exposed by IdentityService to saveUserDetails when the document containing job details is indexed. On job invocation the JobScheduler would then request an access token through the IdentityService and security will lookup the user details in its index that tracks job IDs and associated users when it issues an access token.

[cwperks]

Closing this issue in favor of: #2625

Yesterday I was able to confirm the behavior of job scheduler. The JobSweeper is an IndexOperatorListener which means it listens on indices that JobScheduler expected new job details to be indexed to.

JobScheduler should call on the identity system to persist user info on job details index, delete user info on job details deletion and issue an access token on behalf of the saved user info on job invocation.