Restore primary key field defaults and support for create() #10958

[jacobtylerwalls]

Types of changes

• Bugfix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)
• Breaking change (fix or feature that would cause existing functionality to not work as expected)

Description of Change

Restore model field-level defaults for primary keys.
Closes #10958

Before
Django 3.0 introduced an optimization for models with defaults defined on their primary key fields in a way that prevents "blind overwrites". By that, I mean creating an instance without fetching it from the database, assigning a primary key attribute manually, and saving it back, intending to potentially overwrite data. Arches does blind overwrites (e.g. in management commands with an --overwrite flag).

The stopgap as part of Arches' upgrade to Django 3.2 was to remove field-level defaults altogether and add code to __init__() to set a value, but the problems with that are:

• you can no longer use a model manager create() method without specifying the primary key, which is redundant and surprising. Support for create() is blocking Interface for querying & updating resource models by node & nodegroup aliases #11595.
• any transformations after the instance are created are not captured, e.g. if a model instance is later edited to have .pk = None

After
Field-level defaults are back, plus a solution to support blind overwrites (possible now on Django 5.1+, which supports force_update=True for a blind overwrite). Left a comment discouraging copying this pattern forward to future models.

While here

• Prefer db_default (Django 5+) over our custom migration operations that were auto-creating UUID v1's.
• This migration operation was causing sqlmigrate to crash, see Use db_default for auto-UUID fields in Django 5 #10958
• It also needed to be updated to use uuidv4, so we can just drop our custom operation.
• Change other UUID fields to use uuid v4.

Issues Solved

Closes #10958

Checklist

• I targeted one of these branches:
  • dev/8.0.x (under development): features, bugfixes not covered below
  • dev/7.6.x (main support): regressions, crashing bugs, security issues, major bugs in new features
  • dev/6.2.x (extended support): major security issues, data loss issues
• I added a changelog in arches/releases
• I submitted a PR to arches-docs (if appropriate)
• Unit tests pass locally with my changes
• I added tests that prove my fix is effective or that my feature works
• [n/a] My test fails on the target branch

Ticket Background

• Sponsored by: Getty Conservation Institute
• Found by: @jacobtylerwalls
• Tested by: @jacobtylerwalls

Further comments

The postgres rule checking for bidirectional concept relation uniqueness was preventing using db_default, giving this:

======================================================================
ERROR: setUpClass (tests.exporter.datatype_to_rdf_tests.RDFExportUnitTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/jwalls/django/django/db/models/query.py", line 934, in get_or_create
    return self.get(**kwargs), False
           ~~~~~~~~^^^^^^^^^^
  File "/Users/jwalls/django/django/db/models/query.py", line 632, in get
    raise self.model.DoesNotExist(
        "%s matching query does not exist." % self.model._meta.object_name
    )
arches.app.models.models.Relation.DoesNotExist: Relation matching query does not exist.

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "/Users/jwalls/django/django/db/backends/utils.py", line 105, in _execute
    return self.cursor.execute(sql, params)
           ~~~~~~~~~~~~~~~~~~~^^^^^^^^^^^^^
psycopg2.errors.FeatureNotSupported: cannot perform INSERT RETURNING on relation "relations"
HINT:  You need an unconditional ON INSERT DO INSTEAD rule with a RETURNING clause.

I worked around that by rewriting the rule as a functional UniqueConstraint, replacing the unique constraint we already had. Replacing 1 constraint and 1 rule with 1 constraint is presumably more efficient, but I haven't benched it. I added a test.

[jacobtylerwalls]

Looks like the constraint I added causes instability in the migrations autodetector. This is fixed in 5.2 beta which comes out tomorrow, so I'll just bump that separately tomorrow and then rebase 👍

[chiatt]

Looks like the constraint I added causes instability in the migrations autodetector. This is fixed in 5.2 beta which comes out tomorrow, so I'll just bump that separately tomorrow and then rebase 👍

I noticed that the mixin isn't working with Django 5.1, but it worked fine with 5.2. Should be good to rebase now that #11797 is ready.

[jacobtylerwalls]

I noticed that the mixin isn't working with Django 5.1, but it worked fine with 5.2.

Thanks, that's probably because I'm using the .pk_fields property that was added in 5.2 to be resilient against composite primary keys.

Rebased, ready for another look!

[chiatt]

LGTM!