Fix user filtering in PagerDuty migrator when MIGRATE_USERS=true. When PAGERDUTY_FILTER_USERS is set and MIGRATE_USERS=true, now only the specified users will be migrated. Added filter_users function, updated migrate function, added tests, and updated documentation.

Author: joeyorlando

tools/migrators/README.md

@@ -238,6 +238,11 @@ The migrator will include:
 - Resources associated with USER1 or USER2
 - Resources that match both criteria
 
+Additionally, when `MIGRATE_USERS` is set to `true` and `PAGERDUTY_FILTER_USERS` is specified,
+only the users with the specified PagerDuty IDs will be migrated. This allows for selective user
+migration, which is useful when you want to test the migration with a small set of users before
+migrating all users.
+
 This allows for more flexible and intuitive filtering when migrating specific subsets of your PagerDuty setup.
 
 ### Output Verbosity
@@ -276,8 +281,8 @@ Configuration is done via environment variables passed to the docker container.
 | `EXPERIMENTAL_MIGRATE_EVENT_RULES_LONG_NAMES` | Include service & integrations names from PD in migrated integrations (only effective when `EXPERIMENTAL_MIGRATE_EVENT_RULES` is `true`).                                                                                                                                                                                                                                                                          | Boolean                             | `false` |
 | `MIGRATE_USERS`                               | If `false`, will allow you to important all objects, while ignoring user references in schedules and escalation policies. In addition, if `false`, will also skip importing User notification rules. This may be helpful in cases where you are unable to import your list of Grafana users, but would like to experiment with OnCall using your existing PagerDuty setup as a starting point for experimentation. | Boolean                             | `true`  |
 | `PAGERDUTY_FILTER_TEAM`                       | Filter resources by team name. Resources associated with this team will be included in the migration.                                                                                                                                                                                                                                                                                                          | String                              | N/A     |
-| `PAGERDUTY_FILTER_USERS`                      | Filter resources by PagerDuty user IDs (comma-separated). Resources associated with any of these users will be included in the migration.                                                                                                                                                                                                                                                                             | String                              | N/A     |
-| `PAGERDUTY_FILTER_SCHEDULE_REGEX`             | Filter schedules by name using a regex pattern. Schedules whose names match this pattern will be included in the migration.                                                                                                                                                                                                                                                    | String                              | N/A     |
+| `PAGERDUTY_FILTER_USERS`                      | Filter by PagerDuty user IDs (comma-separated). This serves two purposes: 1) Resources associated with any of these users will be included in the migration, and 2) When `MIGRATE_USERS` is `true`, only these specific users will be migrated (not all users).                                                                                                                                                                             | String                              | N/A     |
+| `PAGERDUTY_FILTER_SCHEDULE_REGEX`             | Filter schedules by name using a regex pattern. Schedules whose names match this pattern will be included in the migration.                                                                                                                                                                                                                                                                    | String                              | N/A     |
 | `PAGERDUTY_FILTER_ESCALATION_POLICY_REGEX`    | Filter escalation policies by name using a regex pattern. Policies whose names match this pattern will be included in the migration.                                                                                                                                                                                                                                                                           | String                              | N/A     |
 | `PAGERDUTY_FILTER_INTEGRATION_REGEX`          | Filter integrations by name using a regex pattern. Integrations whose names match this pattern will be included in the migration.                                                                                                                                                                                                                                                                              | String                              | N/A     |
 | `PRESERVE_EXISTING_USER_NOTIFICATION_RULES`   | Whether to preserve existing notification rules when migrating users                                                                                                                                                                                                                                                                                                                                               | Boolean                             | `true`  |

tools/migrators/lib/pagerduty/migrate.py

@@ -51,6 +51,35 @@
 )
 
 
+def filter_users(users: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
+    """
+    Filter users based on PAGERDUTY_FILTER_USERS.
+
+    When PAGERDUTY_FILTER_USERS is set, only users with IDs in that list will be included.
+    """
+    if not PAGERDUTY_FILTER_USERS:
+        return users  # No filtering, return all users
+
+    filtered_users = []
+    filtered_out = 0
+
+    for user in users:
+        if user["id"] in PAGERDUTY_FILTER_USERS:
+            filtered_users.append(user)
+        else:
+            filtered_out += 1
+
+    if filtered_out > 0:
+        summary = f"Filtered out {filtered_out} users (keeping only users specified in PAGERDUTY_FILTER_USERS)"
+        print(summary)
+
+        # Only print detailed info in verbose mode
+        if VERBOSE_LOGGING:
+            print(f"{TAB}Keeping only users with IDs: {', '.join(PAGERDUTY_FILTER_USERS)}")
+
+    return filtered_users
+
+
 def filter_schedules(schedules: List[Dict[str, Any]]) -> List[Dict[str, Any]]:
     """
     Filter schedules based on configured filters.
@@ -252,6 +281,11 @@ def migrate() -> None:
         users = session.list_all("users", params={"include[]": "notification_rules"})
         oncall_users = OnCallAPIClient.list_users_with_notification_rules()
 
+        # Apply filtering to users if specified
+        if PAGERDUTY_FILTER_USERS:
+            print(f"▶ Filtering users based on PAGERDUTY_FILTER_USERS...")
+            users = filter_users(users)
+
         # Match users with Grafana OnCall users
         for user in users:
             match_user(user, oncall_users)

tools/migrators/lib/tests/pagerduty/test_migrate.py

@@ -4,6 +4,7 @@
     filter_escalation_policies,
     filter_integrations,
     filter_schedules,
+    filter_users,
     migrate,
 )
 
@@ -35,6 +36,51 @@ def test_users_are_skipped_when_migrate_users_is_false(
     mock_oncall_client.list_users_with_notification_rules.assert_not_called()
 
 
+@patch("lib.pagerduty.migrate.MIGRATE_USERS", True)
+@patch("lib.pagerduty.migrate.PAGERDUTY_FILTER_USERS", ["USER1", "USER3"])
+@patch("lib.pagerduty.migrate.MODE", "migrate")  # Skip report generation
+@patch("lib.pagerduty.migrate.APISession")
+@patch("lib.pagerduty.migrate.OnCallAPIClient")
+def test_only_specified_users_are_processed_when_filter_users_is_set(
+    MockOnCallAPIClient, MockAPISession
+):
+    mock_session = MockAPISession.return_value
+
+    # Create test users with required fields
+    users = [
+        {"id": "USER1", "name": "User 1", "oncall_user": None, "email": "[email protected]"},
+        {"id": "USER2", "name": "User 2", "oncall_user": None, "email": "[email protected]"},
+        {"id": "USER3", "name": "User 3", "oncall_user": None, "email": "[email protected]"},
+        {"id": "USER4", "name": "User 4", "oncall_user": None, "email": "[email protected]"}
+    ]
+
+    # Configure mock to return test users for first call, empty lists for other calls
+    mock_session.list_all.side_effect = [
+        users,  # users
+        [],  # schedules
+        [],  # escalation_policies
+        [],  # services
+        [],  # vendors
+    ]
+    mock_session.jget.return_value = {"overrides": []}
+
+    # Mock the user matching function to set oncall_user
+    with patch("lib.pagerduty.migrate.match_user") as mock_match_user:
+        def set_oncall_user(user, _):
+            # Just leave oncall_user as it is (None)
+            pass
+
+        mock_match_user.side_effect = set_oncall_user
+
+        # Run migrate
+        migrate()
+
+        # Check that match_user was only called for USER1 and USER3
+        assert mock_match_user.call_count == 2
+        user_ids = [call_args[0][0]["id"] for call_args in mock_match_user.call_args_list]
+        assert set(user_ids) == {"USER1", "USER3"}
+
+
 class TestPagerDutyFiltering:
     def setup_method(self):
         self.mock_schedule = {
@@ -74,6 +120,26 @@ def setup_method(self):
             },
         }
 
+        self.users = [
+            {"id": "USER1", "name": "User 1"},
+            {"id": "USER2", "name": "User 2"},
+            {"id": "USER3", "name": "User 3"},
+        ]
+
+    @patch("lib.pagerduty.migrate.PAGERDUTY_FILTER_USERS", ["USER1", "USER3"])
+    def test_filter_users(self):
+        """Test filtering users by ID when PAGERDUTY_FILTER_USERS is set."""
+        filtered = filter_users(self.users)
+        assert len(filtered) == 2
+        assert {u["id"] for u in filtered} == {"USER1", "USER3"}
+
+    @patch("lib.pagerduty.migrate.PAGERDUTY_FILTER_USERS", [])
+    def test_filter_users_no_filter(self):
+        """Test that all users are kept when PAGERDUTY_FILTER_USERS is empty."""
+        filtered = filter_users(self.users)
+        assert len(filtered) == 3
+        assert {u["id"] for u in filtered} == {"USER1", "USER2", "USER3"}
+
     @patch("lib.pagerduty.migrate.PAGERDUTY_FILTER_TEAM", "Team 1")
     def test_filter_schedules_by_team(self):
         schedules = [