Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Refactor bulk_create calls #9047

Open
wants to merge 10 commits into
base: develop
Choose a base branch
from
Open

Conversation

zhiltsov-max
Copy link
Contributor

@zhiltsov-max zhiltsov-max commented Feb 3, 2025

Motivation and context

  • Added default batch size in bulk create calls
  • Removed SQLite workaround in bulk_create() calls

How has this been tested?

Checklist

  • I submit my changes into the develop branch
  • I have created a changelog fragment
  • I have updated the documentation accordingly
  • I have added tests to cover my changes
  • I have linked related issues (see GitHub docs)

License

  • I submit my code changes under the same MIT License that covers the project.
    Feel free to contact the maintainers if that's a concern.

@zhiltsov-max zhiltsov-max requested a review from SpecLad as a code owner February 3, 2025 17:58
@SpecLad
Copy link
Contributor

SpecLad commented Feb 4, 2025

What's the purpose of this?

@zhiltsov-max
Copy link
Contributor Author

As the description states - use chunk size by default. It's often unintentionally forgotten.

@SpecLad
Copy link
Contributor

SpecLad commented Feb 4, 2025

As the description states - use chunk size by default. It's often unintentionally forgotten.

Sure, but why do we want this?

@zhiltsov-max
Copy link
Contributor Author

Because we don't want it to be unintentionally forgotten in random places.

@codecov-commenter
Copy link

codecov-commenter commented Feb 4, 2025

Codecov Report

Attention: Patch coverage is 87.09677% with 8 lines in your changes missing coverage. Please review.

Project coverage is 73.92%. Comparing base (908a47d) to head (cc6f205).
Report is 19 commits behind head on develop.

Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #9047      +/-   ##
===========================================
+ Coverage    73.43%   73.92%   +0.48%     
===========================================
  Files          419      428       +9     
  Lines        44351    44536     +185     
  Branches      3875     3881       +6     
===========================================
+ Hits         32569    32922     +353     
+ Misses       11782    11614     -168     
Components Coverage Δ
cvat-ui 77.44% <ø> (+<0.01%) ⬆️
cvat-server 71.01% <87.09%> (+0.90%) ⬆️

Comment on lines 34 to 37
ignore_conflicts: bool = ...,
update_conflicts: bool | None = ...,
update_fields: Sequence[str] | None = ...,
unique_fields: Sequence[str] | None = ...,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can't this be replaced with **kwargs?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It can, I just wanted it to be a "drop-in" replacement with the same signature. It can't be a pure drop-in, because now we require the first model argument, but this design offers good enough compatibility.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Okay, but then why use the ellipses? Seems like you could copy the default values from the original signature as well.

Copy link
Contributor Author

@zhiltsov-max zhiltsov-max Feb 7, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I used annotations from https://github.com/sbdchd/django-types/blob/062b1bfe5313b7e9f1a0a165fa5bf549e6b4addd/django-stubs/db/models/query.pyi#L52 that are used by VS Code, which is our default development environment, and what most developers will typically see in the editor help. The more official django-stubs repo has the same annotations. Ellipses are supposed to mean "unchanged", and this is enforced by the implementation. If the official defaults ever change, we will not have to do any changes in this function.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

and what most developers will typically see in the editor help.

I mean... would you rather see ellipses or actual values?

The more official django-stubs repo has the same annotations.

No it doesn't, check again. 🙂

Ellipses are supposed to mean "unchanged", and this is enforced by the implementation. If the official defaults ever change, we will not have to do any changes in this function.

I think the official defaults changing would imply a backwards compatibility break, so wouldn't it actually be useful for us to have hardcoded them?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

No it doesn't, check again. 🙂

Ok, probably I missed this.

I think the official defaults changing would imply a backwards compatibility break, so wouldn't it actually be useful for us to have hardcoded them?

I think it doesn't matter. Time spent on discussing this question is probably bigger than for changing the code in such case.

It seems Django is quite confident about the values, so I copied them from the "official" stub.

cvat/apps/engine/model_utils.py Outdated Show resolved Hide resolved
cvat/apps/engine/views.py Outdated Show resolved Hide resolved
cvat/apps/engine/model_utils.py Show resolved Hide resolved
cvat/apps/engine/default_settings.py Outdated Show resolved Hide resolved
cvat/apps/engine/model_utils.py Show resolved Hide resolved
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants