Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: reset dataset before every api test #1131

Merged

Conversation

Junjiequan
Copy link
Member

@Junjiequan Junjiequan commented Mar 22, 2024

Description

Added DB collection reset for each API test.

Motivation

API test cases are failing due to shared database usage, where leftover data from previous tests (e.g., datasets or samples) isn't being cleaned up properly.

Tests included/Docs Updated?

  • Included for each change/fix?
  • Passing? (Merge will not be approved unless this is checked)
  • Docs updated?
  • New packages used/requires npm install?
  • Toggle added for new features?

@Junjiequan Junjiequan marked this pull request as ready for review March 22, 2024 14:08
Copy link
Contributor

@bpedersen2 bpedersen2 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good idea

@bpedersen2 bpedersen2 force-pushed the SWAP-3855-scicat-be-tests-add-datasets-deletion-for-all-re branch from d5f863b to 2e8baf6 Compare March 26, 2024 07:21
@Junjiequan Junjiequan force-pushed the SWAP-3855-scicat-be-tests-add-datasets-deletion-for-all-re branch from 2e8baf6 to 2d17761 Compare April 2, 2024 11:20
@Junjiequan Junjiequan merged commit a7a045c into master Apr 2, 2024
7 checks passed
@Junjiequan Junjiequan deleted the SWAP-3855-scicat-be-tests-add-datasets-deletion-for-all-re branch April 2, 2024 11:34
sofyalaski pushed a commit to sofyalaski/scicat-backend-next that referenced this pull request May 23, 2024
* reset dataset before every api test

* Clear collections before each api test
despadam added a commit that referenced this pull request Jul 29, 2024
* first tests

* feat: reset dataset before every api test (#1131)

* reset dataset before every api test

* Clear collections before each api test

* Fixes after jobs testing:
1. Use CreateJobAuth enum instead of AuthOp to define jobDatasetAuthorization
2. Shift a block if groupOwner is present down after the admin check was passed (admin create a job for anonym)
3. Check if current authorization is in jobDatasetAuthorization in instanceAuthorizationJobCreate (types  mismatching)
4. Add a check if unauthorized user creates a job for someone else

* Fixes after jobs testing:
if anonymous user creates a job, initialize a jwt instance for him/her

* Fixes after testing:
string array includes string enum

* dataset(s)Validation redundancy after a merged PR

* test Jobs authorization wip

* fix filtering syntax

* fixes after Job testing:
1. comment out jobConfig matchin, as now in interceptor
2. fix filtering object hierarchy
in #dataset kind of jobConfig's
3. not defining jobInstance ownerUser/Group as empty string
4. add jobInstance.jobParams initialization
5. add a check if datasets passed in the ID list exist

* minor changes

* fix findOne filtering syntax

* add Jobs tests

* remove redundancy

* fix numbering

* extend UPDATE_JOB_GROUP user's right to match docs

* remove unneccessary arguments

* make function return value of findOne  an instance of JobClass

* add statusUpdate tests

* add jobParams to the job instance

* Patch tests

* Fixes after tests:
* Anonymous user can patch status update to a job in #all config. Added instance authorization

* Implement JobDelete authorization

* add deleteJobGroups from env to configuration

* add username for anonymous user when creating/updating status of the job

* statusUpdate instead of update

* Fixes after testing:

* similarly to Datasets, mongoose returns an Object and not JobClass instances. To pass this into CASL we first need  to make those class instances. generateJobInstanceForPermissions implements a copy of object as correct class instance only with properties required in casl.

* functionality of instanceAuthorizationJobStatusUpdate is now implemented in the patch endpoint function. This is done to avoid redefining a found job as a instance of JobClass with all properties

* For both GET endpoints only the endpoint authorisation was implemented for user in the functions of the controller. To add the instance authorization too, first create JobClass instance of the found job Objects and pass them to respective casl expressions.

* adds implementation of delete endpoints. The rules are defined in the CheckPolicies decorator.

* Finalized tests for authorization

* add delete_job_groups definition

* fix unit tests after test jobconfig.json changes

* empty collections before testing

* changes requested in PR

* eremove version duplicate

* change getJobMatchingConfiguration function to explicitly use job type, st it can be used for all cases where we need to extract the configuration, without having to pass the full dto

* run full testing workflow on pushing to release branches

* add env variable for jobs config path for github workflow

* Fix lint errors

- Run lint:fix
- Ignore no-explicit-any rule for jobParams (which are not type checked)
- remove 'read' jobOperation (not implemented)

* Fix lint following merge

* Update test/config/pretest.js to use "dotenv"

Co-authored-by: Jay <[email protected]>

* Fix pretest.js after merging suggestions

* Fix pretest.js

* Fix jobconfig no-explicit-any issues

- Revert previous changes to ignore no-explicit-any
- Use `unknown` for jobParam inputs plus stronger runtime checks
- Fix compilation error in elastic-search service relating to
  deleting non-optional variables. 
- Remove rabbitmqaction validation. RabbitMQ should ignore the DTO body,
  and validate the config itself in the constructor.

* Fix jobconfig no-explicit-any issue (missed logaction)

* changes in tests:
0060 now checks if a job was passed without a required parameter JobParams
0065 (new) checks if a job was passed with an empty object for jobParams
1950 now tries to delete a job that doesn't exist and fails
1960 was called 1950 before
counts on GET are changed (because 0060 before was passing)

* address comments in the PR:
*  now makes one request to the db to find datasets and extracts their pids.
*  loggs the names of all dataset pids that are not in the db in case of an error
*  job endpoint implements the check that the jobParams were passed and are not empty, otherwise it will throw a bad request with respective message
*  job endpoint loggs the id of job to be deleted and implements a check if job exist in db, otherwise throws an error.

---------

Co-authored-by: Jay Quan <[email protected]>
Co-authored-by: Spencer Bliven <[email protected]>
Co-authored-by: Despina Adamopoulou <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants