Dual Write: Generate replication events for Role endpoints #1172

lpichler · 2024-08-20T12:18:26Z

Class RelationApiDualWriteHandler was added. This class containers has methods which performs Replication Event generation
methods:
- generate_relations_from_current_state_of_role - this generate relations from current stage of role only from PRBAC DB- it uses mappings tables to get v2 role and role bindings uuids
- generate_relations_and_mappings_for_role - this generates new relations for role - it generates new UUID for v2 role and role bindings
- regenerate_relations_and_mappings_for_role - generate_relations_and_mappings_for_role
- build_replication_event - generates json representation of relation which are sent to outbox table - see replication event bellow
- save_replication_event_to_outbox - store relation event into outbox table (content of this method is not added yet)
- generate_replication_event_to_outbox - regenerate_relations_and_mappings_for_role and save_replication_event_to_outbox

Each role action has mix of those methods:
CREATE:

create role from input to DB
generate_relations_and_mappings_for_role
save_replication_event_to_outbox

UPDATE:

generate_relations_from_current_state_of_role
update role based on input in DB
generate_replication_event_to_outbox

DELETE:

generate_relations_from_current_state_of_role
delete role in DB
save_replication_event_to_outbox

Replication Event is not stored into outbox table, PR with table is not merged yet- PR
Migration tool and mappings update: Columns v2 role uuid and permissions where added into BindingMapping table ,
this information are required to identification of role and role bindings
Control Variable REPLICATION_TO_RELATION_ENABLED to enabled and disable replication - Replication is disable by default except to unit tests

Next Steps: Missing Endpoints

Replication Event (which will be stored in outbox table as payload):

{
  "relations_to_add": [
    {
      "resource": {
        "type": "role_binding",
        "id": "049fd753-4fe2-4d70-bc56-c59a68d749df"
      },
      "relation": "granted",
      "subject": {
        "type": "role",
        "id": "a1cb1ab4-0443-4257-b576-2eec75079d3d"
      }
    },
    {
      "resource": {
        "type": "role",
        "id": "a1cb1ab4-0443-4257-b576-2eec75079d3d"
      },
      "relation": "app_all_read",
      "subject": {
        "type": "user",
        "id": "*"
      }
    },
  ],
  "relations_to_remove": [
    {
      "resource": {
        "type": "keya/id",
        "id": "valueA"
      },
      "relation": "user_grant",
      "subject": {
        "type": "role_binding",
        "id": "bf8e6f6b-19b6-446a-81e4-d48a9b694bd4"
      }
    }]
}

Links

https://issues.redhat.com/browse/RHCLOUD-34786
https://issues.redhat.com/browse/RHCLOUD-34509

lpichler · 2024-08-22T17:12:46Z

/retest

lennysgarage · 2024-08-22T17:19:02Z

rbac/management/role/view.py

@@ -371,7 +394,7 @@ def update(self, request, *args, **kwargs):
        @apiParam (Path) {String} id Role unique identifier

        @apiParam (Request Body) {String} name Role name
-        @apiParam (Request Body) {Array} access Access definition
+        @apiParam (Request Body) {ArRray} access Access definition


Might be a small nitpick not sure if the docstrings affect things here but should ArRray just be Array?

alechenninger · 2024-08-26T21:45:58Z

rbac/management/role/model.py

    id = models.UUIDField(default=uuid4, primary_key=True)
    v1_role = models.ForeignKey(Role, on_delete=models.CASCADE)
+    v2_role = models.ForeignKey(V2Role, on_delete=models.CASCADE)
+    permissions = ArrayField(models.CharField(max_length=200), blank=True, null=True)


Also note that since most operations are performed in their own isolated transaction (autocommit / not inside atomic()) then I think the perf overhead of repeatable read (or serializable for that matter) should be negligible (however you will also not see any of the benefits of the increased isolation level in those cases–it will only matter to atomic() blocks).

Option 2 was added.

Fixes RHCLOUD-35563

…erified

Assumes eventual API change from IT API

[RHCLOUD-35227] Replicate group removal

…rkspace-relations Create proper root/default workspaces on initial migration

…ples RHCLOUD-35448: Start using Principal.user_id for group#member@principal tuples

alechenninger · 2024-10-10T13:01:34Z

rbac/management/group/relation_api_dual_write_group_handler.py

+
+    def prepare_to_delete_group(self):
+        """Generate relations to delete."""
+        roles = Role.objects.filter(policies__group=self.group)


Need to add a check if replication is enabled here

lpichler · 2024-10-11T12:07:01Z

/retest

…ndler

…dual write handler

alechenninger

IMO this is ready to ship

lpichler force-pushed the dual_write_for_role branch 12 times, most recently from 34abb11 to 9b5a83f Compare August 21, 2024 16:11

lpichler changed the title ~~[WIP] Dual Write: Generate replication event for Role endpoints~~ Dual Write: Generate replication event for Role endpoints Aug 21, 2024

lpichler requested review from astrozzc and coderbydesign August 21, 2024 16:29

lpichler changed the title ~~Dual Write: Generate replication event for Role endpoints~~ Dual Write: Generate replication events for Role endpoints Aug 21, 2024

lpichler force-pushed the dual_write_for_role branch 10 times, most recently from a8185c5 to 88ba65f Compare August 22, 2024 12:20

lennysgarage reviewed Aug 22, 2024

View reviewed changes

lpichler mentioned this pull request Aug 26, 2024

Don't create root workspace in database in migration tool #1176

Merged

alechenninger reviewed Aug 26, 2024

View reviewed changes

coderbydesign and others added 17 commits October 4, 2024 14:04

Update role view tests to reference workspace

711e411

Update group view tests to use workspace reference

5d80684

Use correct user_id for user, and replicate with domain prefix

96788f3

Fixes RHCLOUD-35563

Undo rename

af81cab

Undo another rename

e592de8

Update dual write test asserts for new ID format

df53840

Revert getting user_id in get_principal since principals not always v…

9fda7c6

…erified

Account for lazy ID load and also for service accounts

d45b834

Assumes eventual API change from IT API

Only update the mapping if we're not going to delete it

07f27a8

Fix lint issue

0204e6a

Update group dual-writes to use default workspace for system roles

794bde8

Add some additional tests

4ed6230

Resolve failing tests needing built-in workspaces

6a95813

Merge pull request #1222 from RedHatInsights/remove_groups_1

6d15f05

[RHCLOUD-35227] Replicate group removal

Merge pull request #1220 from coderbydesign/RHCLOUD-35292-built-in-wo…

44ac90f

…rkspace-relations Create proper root/default workspaces on initial migration

Merge pull request #1224 from RedHatInsights/rhcloud-35448-user-id-tu…

19dbbb4

…ples RHCLOUD-35448: Start using Principal.user_id for group#member@principal tuples

Fix & unskip tests post merge

d97b639

alechenninger reviewed Oct 10, 2024

View reviewed changes

lpichler and others added 9 commits October 11, 2024 14:33

Add guard condition in prepare_to_delete_group in group dual write ha…

2fbc193

…ndler

Remove unecessary print and change type->event in replicatin event

2543055

Merge branch 'master' into dual_write_for_role

3e5a62a

Add guard condition in replicate_new_system_role_permissions in role …

5cc27e3

…dual write handler

Remove unused method

41cf9c7

Rename methods per prior discussion

83d77f1

Simplify how config works for excluding resource definition migration

9b3aea7

Undo formatting changes

9bec537

Update description

09cb3d4

alechenninger approved these changes Oct 11, 2024

View reviewed changes

alechenninger merged commit cfea2f6 into master Oct 11, 2024
9 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Dual Write: Generate replication events for Role endpoints #1172

Dual Write: Generate replication events for Role endpoints #1172

lpichler commented Aug 20, 2024 •

edited

Loading

lpichler commented Aug 22, 2024

lennysgarage Aug 22, 2024

This comment was marked as resolved.

alechenninger Aug 26, 2024

lpichler Aug 29, 2024

alechenninger Oct 10, 2024

lpichler commented Oct 11, 2024

alechenninger left a comment

Dual Write: Generate replication events for Role endpoints #1172

Dual Write: Generate replication events for Role endpoints #1172

Conversation

lpichler commented Aug 20, 2024 • edited Loading

Links

lpichler commented Aug 22, 2024

lennysgarage Aug 22, 2024

Choose a reason for hiding this comment

This comment was marked as resolved.

alechenninger Aug 26, 2024

Choose a reason for hiding this comment

lpichler Aug 29, 2024

Choose a reason for hiding this comment

alechenninger Oct 10, 2024

Choose a reason for hiding this comment

lpichler commented Oct 11, 2024

alechenninger left a comment

Choose a reason for hiding this comment

lpichler commented Aug 20, 2024 •

edited

Loading