Release v2.32.0 #9248
Merged
Release v2.32.0 #9248
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![cvat-bot[bot]](https://avatars.githubusercontent.com/in/406724?s=60&v=4){kind=small}
Update develop after v2.31.0
All of the support code in the auto-annotation SDK layer is already there, we just need to send sublabel information when creating a native function, and validate it when running an agent. In addition, force keypoint sublabel types to be "points", since this is how the UI creates them for tasks/projects.
This PR adds a search and filtering features to Organization page.
Removed extra styles that caused extra scrolls to appear
This PR specifies explicitly which arguments are expected to be `positional-only` or `keyword-only` in methods decorated with `validate_file_status`. This fixes the newly introduced issue when `IndexError` is raised when checking file status [1]. [1] https://github.com/cvat-ai/cvat/blob/6cefd3fe0025d58236de9db38f12622d889518e4/cvat/apps/engine/cloud_provider.py#L123
Continuing to remove Black exceptions.
Coverage of consensus task creation from #8939 ### How has this been tested? - Check `Advanced Configuration` -> `Consensus Replicas` field in task creation. Allowed values: `0 <= x <= 10` but `x≠1`. Bad values should trigger form to show error message. Creating task with unmet condition should result in an error notification - Submit & Open task - Task page should contain two tags with text 'Consensus' - Consensus job should have a collapsible drop-down with text e.g. `2 Replicas`, the number should be correct - Open drop-down with job replicas. There should be a correct number of replicas, ordered desc by job ids, top to bottom ### Checklist - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: cvat-bot[bot] <147643061+cvat-bot[bot]@users.noreply.github.com> Co-authored-by: Oleg Valiulin <[email protected]> Co-authored-by: Kirill Lakhov <[email protected]>
This makes them easier to read, as the key is visually separated from the value. Also, it emphasizes the key-value nature of labels.
This PR introduces the following changes:
- Several server API endpoints have been deprecated or no longer handle
the process of exporting specific resources:
- [API] **GET /api/projects/id/dataset?action=import_status** is
deprecated
- [API] **GET /api/projects/id/dataset**(no action parameter or action
!= import_status) returns 410 status code - API endpoint no longer used
to handle export datasets flow
- [API] **GET /api/projects/id/annotations** returns 410 status code -
API endpoint no longer used to handle export annotations flow
- [API] **GET /api/projects/id/backup** returns 410 status code - API
endpoint no longer used to handle export backups flow
- [API] **GET /api/tasks/id/backup** returns 410 status code - API
endpoint no longer used to handle export backups flow
- [API] **GET /api/tasks/id/annotations?format=<format>** returns 410
status code - API endpoint no longer used to handle export annotations
flow
- [API] **GET /api/tasks/d/dataset** returns 410 status code - API
endpoint no longer used to handle export datasets flow
- [API] **GET /api/jobs/id/annotations?format=<format>** returns 410
status code - API endpoint no longer used to handle export annotations
flow
- [API] **GET /api/jobs/id/dataset** returns 410 status code - API
endpoint no longer used to handle export datasets flow
- Introduced new "private" endpoints to download prepared files (they
are hidden in the generated server schema):
- [API] GET /api/projects/id/dataset/download?rq_id=rq_id
- [API] GET /api/projects/id/backup/download?rq_id=rq_id
- [API] GET /api/tasks/id/dataset/download?rq_id=rq_id
- [API] GET /api/tasks/id/backup/download?rq_id=rq_id
- [API] GET /api/jobs/id/dataset/download?rq_id=rq_id
- Changed permissions used when downloading a prepared file (now admin
and RQ job owner have rights to download a prepared file related to a
specific background job (it's a questionable topic whether should be
resource access checked also))
- Added result filename saving to RQ metadata for export-related RQ
jobs:
```
{
...
"result_filename": str // is used as the final filename when downloading files/uploading to cloud storage
}
```
- Final file name is taken from meta when uploading a file to cloud
storage (`key` and `key_pattern` args have been removed)
#### Depends on #9178 ### Motivation and context Test coverage for merging feature in consensus jobs (#8953) Commits were cherry-picked from #9190 ### How has this been tested? ### Case 1: check new merge buttons exist and are visible. New freshly created jobs have status 'new'. Trying to merge new jobs should result in error notifications and a HTTP 400 status code. All notifications are closed before the next testcase ### Case 2: Check consensus management page Consensus management page has settings for consensus quorum and minimum annotation overlap. Check that the page has fields for both options. If a field is filled incorrectly the form turns red and a message appears under the form. Trying to save options with erroneous forms should do nothing - no requests are sent, no errors shown. ### Case 3: Create annotations and check that job replicas merge correctly Create annotations with one rectangle in job replicas. The rectangle is moved with a step in each job's first frame. This means that after merging, the consensus job will have a rectangle which has to fully match a rectangle inside the replica in the middle. - Create annotations. - Merge consensus jobs. No errors should appear. A notification message validates the merge, close it - Go inside consensus job, remember the rectangle's positions - Go back to the task page using browser's 'Back' button - After loading the task page, consensus job's status should read **competed** - Open the middle job replica. Rectangle should be identical to the previous one ### Checklist - [ ] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Oleg Valiulin <[email protected]> Co-authored-by: Kirill Lakhov <[email protected]>
There is no point in adding "nuclio" to `no_proxy` by default. * During the build, Nuclio isn't used. * When deploying with Docker Compose, the Compose file overrides `no_proxy`. * When deploying with Helm, Nuclio has a different hostname. Even if you're deploying manually, you still have to override `no_proxy`, since there are many other internal hosts besides `nuclio` (like `opa`).
The Traefik 2.x release series goes EOL in under two months (and the 2.10 minor that we currently use has reached EOL more than a year ago). Upgrading will let us keep getting security fixes. In addition, I'm currently working on a patch that would have to be changed after the Traefik upgrade. So I'd rather do the upgrade now and not have to update my patch again.
Worker configuration is still present after #9174 causing an error in vscode during debug config launch
This PR fixes an issue where navigating to task analytics caused `#overview` to be appended to the URL, requiring two back button presses to return to the previous page.
Using rapidjson instead of json-stream to dump annotations in a stream manner on backup
…lows (#9232) Signed-off-by: dependabot[bot] <[email protected]> Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
…9214) I have a need to add an environment variable for backend containers in the Enterprise version. `additionalEnv` doesn't work very well for this, because: * It is an array, so if a user specifies a different value for it with `--set`, any defaults set by the superchart will disappear. * It is inserted into the resource definition verbatim, so you can't refer to things like the release name. This patch introduces a separate setting for this purpose. It takes a map, and runs the variable values through `tpl` to support interpolation.
Small grammatical improvement of interactor message
The current version is incompatible with Python 3.12, which has broken the Helm CI job. Why the job worked up until this point is unclear.
The merged commands are more efficient, safer (because pip will verify that there are no dependency conflicts), and in CI it's easier to understand the output of one pip command than many. Also, in the CI-nightly workflow, separate the dependency install commands into their own step.
Currently, when backup files are extracted from zip file, they first are read to RAM, and then are written to filesystem. It can be a problem, when backup has some large files, e.g. videos. Fixing it.
Test coverage for #9151 ### Motivation and context This tests search within organizations feature ### How has this been tested? New organization pipeline is updated with two test cases: searching and filtering organization members. Alongside this, the suite's setup and teardown routines were updated to use less UI. Improved test suite's running time by 1m (30%) ### Checklist - [x] I submit my changes into the `develop` branch - [ ] I have created a changelog fragment - [ ] I have updated the documentation accordingly - [ ] I have added tests to cover my changes - [ ] I have linked related issues (see [GitHub docs]( https://help.github.com/en/github/managing-your-work-on-github/linking-a-pull-request-to-an-issue#linking-a-pull-request-to-an-issue-using-a-keyword)) ### License - [ ] I submit _my code changes_ under the same [MIT License]( https://github.com/cvat-ai/cvat/blob/develop/LICENSE) that covers the project. Feel free to contact the maintainers if that's a concern. --------- Co-authored-by: Oleg Valiulin <[email protected]>
…/api/requests` (#9236) This PR addresses the `KeyError` by making user/request fields in the meta nullable.
cvat-bot
bot
requested review from
SpecLad,
azhavoro,
bsekachev,
zhiltsov-max and
nmanovic
as code owners
|
|
Codecov ReportAttention: Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #9248 +/- ##
==========================================
+ Coverage 73.33% 74.16% +0.83%
==========================================
Files 448 428 -20
Lines 45868 44964 -904
Branches 3915 3917 +2
==========================================
- Hits 33637 33349 -288
+ Misses 12231 11615 -616
🚀 New features to boost your workflow:
|
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Added
Added parameter
conv_mask_to_polysupport for importing annotations in projects,tasks and jobs (Add option to SDK allowing upload of annotation as masks instead of polygon. #8823)[SDK] Auto-annotation functions that output skeletons can now be used via agents (Add skeleton support for native functions #9122)
Search bar and filtering components on the organization page (Added search to organization page #9151)
Error notification if something is wrong in quality/consensus settings (Fixed styles on Consensus management page #9178)
[Helm] Added a new value,
cvat.backend.extensionEnv, to support supercharts adding environment variables to backend containers (Helm: support adding backend environment variables from supercharts #9214)Timestamps to Uvicorn stdout logs (Add asctime to uvicorn stdout logs #9210)
Added robots.txt file to manage crawling traffic (Added robots.txt file #9241)
Changed
[SDK]
DetectionFunctionSpecnow requires that the type of keypoint sublabels is set topoints. Accordingly,keypoint_specnow sets this type by default (Add skeleton support for native functions #9122)Optimized memory usage on export with YOLO and COCO formats for tasks (Streaming export for YOLO and COCO formats #9084)
Optimized memory usage for project export in YOLO and COCO formats (Dl/stream export project #9159)
Updated Traefik to v3.3.x (Upgrade Traefik to v3 #9202)
[Compose] Traefik access log is now limited to the same fields as in Helm-based deployments (Upgrade Traefik to v3 #9202)
Deprecated
GET /api/projects/id/dataset?action=import_statusAPI endpoint to check the status of the import process. Instead, theGET /api/requests/rq_idrequests API should be used (API to download export results #9075)Removed
GET /api/projects/id/datasetAPI endpoint no longer handles dataset export process (API to download export results #9075)GET /api/projects/id/annotationsAPI endpoint no longer handles annotations export process (API to download export results #9075)GET /api/projects/id/backupAPI endpoint no longer handles project export process (API to download export results #9075)GET /api/tasks/id/datasetAPI endpoint no longer handles dataset export process (API to download export results #9075)GET /api/tasks/id/annotations?format=API endpoint no longer handles annotations export process (API to download export results #9075)GET /api/tasks/id/backupAPI endpoint no longer handles task export process (API to download export results #9075)GET /api/jobs/id/datasetAPI endpoint no longer handles dataset export process (API to download export results #9075)GET /api/jobs/id/annotations?format=API endpoint no longer handles annotations export process (API to download export results #9075)Existing implementation of analytics reports (Removed analytics reports #9174)
Fixed
Removed extra sliders on quality control page (Fix extra sliders on the GT management pages #9168)
Server returns a 404 status code with details instead of a 500 when cloud storage preview defined by a manifest cannot be downloaded ([
engine/cloud_provider.py] Normalize function signatures #9170)Broken styles on Consensus management page (Fixed styles on Consensus management page #9178)
[Helm] Fixed frontend deployment template issue that caused rendering to fail if additional volumes and volume mounts were defined. (Fix frontend helm template #9181)
Incorrect behavior of standard browser back button on the quality page (Prevent extra tab from adding in history #9039)
Fixed a 500 status code that could occur when listing requests by GET /api/requests (Fixed 500 status code that could occur when listing requests by
GET /api/requests#9236)Memory usage optimization on backup import (dont use ram while extracting backup files from zip #9227)