Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add create schedule functionality to scheduler api #3353

Merged
merged 22 commits into from
Aug 22, 2024

Conversation

jpbruinsslot
Copy link
Contributor

@jpbruinsslot jpbruinsslot commented Aug 13, 2024

Context

Adds the functionality of creating a schedule for an arbitrary task through the scheduler API. This functionality is needed for the upcoming recurring report tasks. It is necessary to specify at what interval this task needs to be executed for an individual report task. A schedule for a task can be created as follows:

POST /schedules
(
    "scheduler_id": "id-of-the-scheduler",
    "schedule": "cron-expression",
    "data": {"place": "your", "task": "here"}
}

Changes

  • models/schedule.py
    • computed field for deadline; we want to be able to calculate the deadline for the schedule when we create a schedule from the create endpoint. This allows the API client just to supply the cron expression and not do the deadline calculation itself. Update: a post init to calculate the deadline_at when it isn't supplied and a schedule is present did the job better with less code.
  • server/
    • schedules.py added create endpoint for schedules, adds validation of data by referencing the scheduler with scheduler_id, it also generates the hash of the contents of the data field so this doesn't have to be done by the client.
    • serializers/schedule.py adding ScheduleCreate model to correctly serialize/validate the client request. Take note that issues are already created to tackle the server API like: Refactor scheduler rest api server #3162

QA notes

We're going to create a Schedule through the newly created /schedules endpoint:

  • make reset
  • Fire up openkat
  • Create an organisation called dev. Don't start any scans though!
  • curl -X POST -H "Content-Type: application/json" -d '{"scheduler_id": "boefje-dev", "schedule": "*/5 * * * *", "data": {"id": "12c3f698-5036-40ce-a1d5-1d59e50116d8", "boefje": { "id": "dns-records", "name": "DnsRecords", "version": null }, "input_ooi": "Hostname|internet|mispo.es", "dispatches": [], "organization": "dev" }}' http://localhost:8004/schedules
  • curl HTTP://localhost:8004/schedules it should now list a schedule

Code Checklist

  • All the commits in this PR are properly PGP-signed and verified.
  • This PR only contains functionality relevant to the issue.
  • I have written unit tests for the changes or fixes I made.
  • I have checked the documentation and made changes where necessary.
  • I have performed a self-review of my code and refactored it to the best of my abilities.
  • Tickets have been created for newly discovered issues.
  • For any non-trivial functionality, I have added integration and/or end-to-end tests.
  • I have informed others of any required .env changes files if required and changed the .env-dist accordingly.
  • I have included comments in the code to elaborate on what is not self-evident from the code itself, including references to issues and discussions online, or implicit behavior of an interface.

Checklist for code reviewers:

Copy-paste the checklist from the docs/source/templates folder into your comment.


Checklist for QA:

Copy-paste the checklist from the docs/source/templates folder into your comment.

@jpbruinsslot jpbruinsslot added the mula Issues related to the scheduler label Aug 13, 2024
@jpbruinsslot jpbruinsslot self-assigned this Aug 13, 2024
@jpbruinsslot jpbruinsslot marked this pull request as ready for review August 14, 2024 15:18
@jpbruinsslot jpbruinsslot requested a review from a team as a code owner August 14, 2024 15:18
Copy link
Contributor

@ammar92 ammar92 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nice work. Just a few remarks that might improve it even more. Can be QA'd

mula/scheduler/models/schedule.py Outdated Show resolved Hide resolved
mula/scheduler/models/schedule.py Outdated Show resolved Hide resolved
mula/scheduler/server/handlers/schedules.py Outdated Show resolved Hide resolved
mula/scheduler/server/handlers/schedules.py Outdated Show resolved Hide resolved
@stephanie0x00
Copy link
Contributor

Checklist for QA:

  • I have checked out this branch, and successfully ran a fresh make reset.
  • I confirmed that there are no unintended functional regressions in this branch:
    • I have managed to pass the onboarding flow
    • Objects and Findings are created properly
    • Tasks are created and completed properly
  • I confirmed that the PR's advertised feature or hotfix works as intended.
  • I checked the logs for errors and/or warnings and made issues where necessary

What works:

Seems to look good. Tested with the QA notes and also tested a migration scenario by going through the onboarding and then do an update to the schedule query.

What doesn't work:

n/a

Bug or feature?:

n/a

@underdarknl underdarknl merged commit 929b443 into main Aug 22, 2024
9 checks passed
@underdarknl underdarknl deleted the feature/mula/create-schedule branch August 22, 2024 18:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
mula Issues related to the scheduler
Projects
Archived in project
Development

Successfully merging this pull request may close these issues.

4 participants