Code

Create Databricks access connector#1376

Code

Closed

JCZuurmond wants to merge 124 commits intomaindatabrickslabs/ucx:mainfrom feature/handle-access-connector-managed-identitydatabrickslabs/ucx:feature/handle-access-connector-managed-identityCopy head branch name to clipboard

+765-101

Commits

Commits on Apr 11, 2024

• Add AccessConnector dataclass

  JCZuurmond

  committed
  18618c0

  Copy full SHA for 18618c0

• Move azure resources into seprate fixture

  JCZuurmond

  committed
  b70eb1a

  Copy full SHA for b70eb1a

• Move azure resources fixture into conftest

  JCZuurmond

  committed
  2b7b411

  Copy full SHA for 2b7b411

• Add AccessConnectorClient to resources

  JCZuurmond

  committed
  db9a6c4

  Copy full SHA for db9a6c4

• Add test resources

  JCZuurmond

  committed
  e987e90

  Copy full SHA for e987e90

Commits on Apr 12, 2024

• Add whiteline

  JCZuurmond

  committed
  6065333

  Copy full SHA for 6065333

• Add whiteline

  JCZuurmond

  committed
  9b06464

  Copy full SHA for 9b06464

• Parse accessconnector response

  JCZuurmond

  committed
  91b595c

  Copy full SHA for 91b595c

• Add subscription id and resource group name parsing

  JCZuurmond

  committed
  12a6643

  Copy full SHA for 12a6643

• Implement create access connector

  JCZuurmond

  committed
  b35bdec

  Copy full SHA for b35bdec

• Add api_version to delete method

  JCZuurmond

  committed
  018fbc1

  Copy full SHA for 018fbc1

• Implement delete access connector

  JCZuurmond

  committed
  8214bf0

  Copy full SHA for 8214bf0

• Add TODO about create access connector

  JCZuurmond

  committed
  82dde54

  Copy full SHA for 82dde54

• Refactor AccessConnector to remove name

  JCZuurmond

  committed
  69ab93a

  Copy full SHA for 69ab93a

• Make AccessConnector.type static

  JCZuurmond

  committed
  4533725

  Copy full SHA for 4533725

• Move tests to unit tests

  JCZuurmond

  committed
  4043bcb

  Copy full SHA for 4043bcb

• Merge access connector with existing test setup

  JCZuurmond

  committed
  7962e74

  Copy full SHA for 7962e74

• Add tags to access conenctor

  JCZuurmond

  committed
  6c9acde

  Copy full SHA for 6c9acde

• Fix create access connector

  JCZuurmond

  committed
  ff5200f

  Copy full SHA for ff5200f

• Broaden access connector regex

  JCZuurmond

  committed
  97ee478

  Copy full SHA for 97ee478

• Add todo about list with resource group

  JCZuurmond

  committed
  43687a6

  Copy full SHA for 43687a6

• Add pagination

  JCZuurmond

  committed
  f907a44

  Copy full SHA for f907a44

• Add skipped integration test

  Show description for bd9dbc9
  JCZuurmond

  committed
  bd9dbc9

  Copy full SHA for bd9dbc9

• Add tags when creating access connector

  JCZuurmond

  committed
  1e299a4

  Copy full SHA for 1e299a4

• Refactor method to create or update

  JCZuurmond

  committed
  ac71445

  Copy full SHA for ac71445

• Change back credentials and conftest

  JCZuurmond

  committed
  fc8b98e

  Copy full SHA for fc8b98e

• make a working list access connector API

  nfx

  committed
  5563b66

  Copy full SHA for 5563b66

• Add test for getting access connector

  JCZuurmond

  committed
  a18a78d

  Copy full SHA for a18a78d

• Add get method

  JCZuurmond

  committed
  15d7eef

  Copy full SHA for 15d7eef

• Update get access connector tests

  JCZuurmond

  committed
  da28404

  Copy full SHA for da28404

• Use RawResource for parsing

  JCZuurmond

  committed
  5469968

  Copy full SHA for 5469968

• Set tags default to field with dict factory

  JCZuurmond

  committed
  5799e6d

  Copy full SHA for 5799e6d

• Type hint and reorder

  JCZuurmond

  committed
  47cd89e

  Copy full SHA for 47cd89e

• Fix creation AccessConnector

  JCZuurmond

  committed
  d070318

  Copy full SHA for d070318

• Fix delete access connector

  JCZuurmond

  committed
  4daabff

  Copy full SHA for 4daabff

• Add integration test

  JCZuurmond

  committed
  7e271da

  Copy full SHA for 7e271da

• Fix mypy and fmt

  JCZuurmond

  committed
  e15da94

  Copy full SHA for e15da94

• Change signature to create access connector

  JCZuurmond

  committed
  7315d38

  Copy full SHA for 7315d38

• Change signature to delete access connector

  JCZuurmond

  committed
  a00e2ab

  Copy full SHA for a00e2ab

• Rename resource_group to resource_group_name

  JCZuurmond

  committed
  02234c5

  Copy full SHA for 02234c5

• Add get to access connector integration test

  JCZuurmond

  committed
  a7c52f5

  Copy full SHA for a7c52f5

• Let test reuse access_connector object

  JCZuurmond

  committed
  356ff18

  Copy full SHA for 356ff18

• Extract all access connector attributes

  JCZuurmond

  committed
  ee210d2

  Copy full SHA for ee210d2

• Move get access connector to AzureRources

  JCZuurmond

  committed
  177c02b

  Copy full SHA for 177c02b

• Move list access connectors to AzureResources

  JCZuurmond

  committed
  0aa728d

  Copy full SHA for 0aa728d

• Move create or update access connector to AzureResources

  JCZuurmond

  committed
  bb61ecd

  Copy full SHA for bb61ecd

• Move delete access connector to AzureResources

  JCZuurmond

  committed
  d5bf5d7

  Copy full SHA for d5bf5d7

• Delete AccessConnectorClient

  JCZuurmond

  committed
  97ab633

  Copy full SHA for 97ab633

• Remove TODO on resource group

  JCZuurmond

  committed
  7a633eb

  Copy full SHA for 7a633eb

• Fix access connector integration test

  JCZuurmond

  committed
  bbc3d84

  Copy full SHA for bbc3d84

• Remove old import

  JCZuurmond

  committed
  c9feec5

  Copy full SHA for c9feec5

• Use default_factory instead of default

  JCZuurmond

  committed
  1f4ff16

  Copy full SHA for 1f4ff16

• Format

  JCZuurmond

  committed
  ca01f15

  Copy full SHA for ca01f15

• Make api_version key word arguments

  JCZuurmond

  committed
  b6897db

  Copy full SHA for b6897db

• Use raw resources when parsing the response

  JCZuurmond

  committed
  5689a91

  Copy full SHA for 5689a91

• Update test name

  JCZuurmond

  committed
  ca58ca5

  Copy full SHA for ca58ca5

Commits on Apr 15, 2024

• Move SP to seperate method

  JCZuurmond

  committed
  351d654

  Copy full SHA for 351d654

• Add run access connector migration results

  JCZuurmond

  committed
  661c5fe

  Copy full SHA for 661c5fe

• Add plan confirmed for access connector migration

  JCZuurmond

  committed
  60d836c

  Copy full SHA for 60d836c

• Add integration test for storage accounts

  JCZuurmond

  committed
  cd8455a

  Copy full SHA for cd8455a

• Add and use storage account data class

  JCZuurmond

  committed
  20b042a

  Copy full SHA for 20b042a

• Create access connector for each storage account

  JCZuurmond

  committed
  70523f2

  Copy full SHA for 70523f2

• Let apply storage permission user StorageAccount dataclass

  JCZuurmond

  committed
  5828bee

  Copy full SHA for 5828bee

• Use StorageAccount dataclass

  JCZuurmond

  committed
  1d5046d

  Copy full SHA for 1d5046d

• Add from raw resources class method to AccessConnector

  JCZuurmond

  committed
  9ffa8ed

  Copy full SHA for 9ffa8ed

• Let AccessConnector.id be RawResource

  JCZuurmond

  committed
  d43869b

  Copy full SHA for d43869b

• Remove AccessConnector.type

  JCZuurmond

  committed
  4db3aea

  Copy full SHA for 4db3aea

• Parse identity in RawResource

  JCZuurmond

  committed
  5bf637a

  Copy full SHA for 5bf637a

• Raise ValueErrors when id, name or location are missing in AccessConnector

  JCZuurmond

  committed
  4149017

  Copy full SHA for 4149017

• Handle ValueError when parssing raw access connector

  JCZuurmond

  committed
  1939609

  Copy full SHA for 1939609

• Let StorageAccount.id be AzureResource

  JCZuurmond

  committed
  be23649

  Copy full SHA for be23649

• Update user assigned identity ValueError

  JCZuurmond

  committed
  4da3005

  Copy full SHA for 4da3005

• Change ValueError to KeyError

  JCZuurmond

  committed
  7c946f0

  Copy full SHA for 7c946f0

• Repeat declaring query

  JCZuurmond

  committed
  f2956f2

  Copy full SHA for f2956f2

• Remove redundant str

  JCZuurmond

  committed
  d58f338

  Copy full SHA for d58f338

• Add from_raw_resource to StorageAccount

  JCZuurmond

  committed
  c7fcb0d

  Copy full SHA for c7fcb0d

• Handle KeyError in StorageAccount from_raw_resource

  JCZuurmond

  committed
  bed01e1

  Copy full SHA for bed01e1

• Format

  JCZuurmond

  committed
  b3dc41a

  Copy full SHA for b3dc41a

• Fix log message

  JCZuurmond

  committed
  7e0c85d

  Copy full SHA for 7e0c85d

• Fix mypy on query

  JCZuurmond

  committed
  94b038f

  Copy full SHA for 94b038f

• Fix mypy

  JCZuurmond

  committed
  26c417c

  Copy full SHA for 26c417c

• Moke confirm create access connectors for each storage account

  JCZuurmond

  committed
  84e0910

  Copy full SHA for 84e0910

• Update test to use StorageAccount dataclass

  JCZuurmond

  committed
  c82f107

  Copy full SHA for c82f107

• Mock prompts in test

  JCZuurmond

  committed
  521e140

  Copy full SHA for 521e140

• Let create or update access connector return access connector

  JCZuurmond

  committed
  827ea96

  Copy full SHA for 827ea96

• Apply storage permissions for created access connectors

  JCZuurmond

  committed
  d54241f

  Copy full SHA for d54241f

• Format

  JCZuurmond

  committed
  c6bc001

  Copy full SHA for c6bc001

• Fix return type

  JCZuurmond

  committed
  7b592ab

  Copy full SHA for 7b592ab

• Merge for-loops

  JCZuurmond

  committed
  8d55e2c

  Copy full SHA for 8d55e2c

• Parralelize creation of access connectors

  JCZuurmond

  committed
  b966ee4

  Copy full SHA for b966ee4

• Add wait for provisioning of access connector

  JCZuurmond

  committed
  b779d49

  Copy full SHA for b779d49

• Shorten variable name

  JCZuurmond

  committed
  2ffc308

  Copy full SHA for 2ffc308

• Format

  JCZuurmond

  committed
  b6b1599

  Copy full SHA for b6b1599

• Allow two letter attributes

  JCZuurmond

  committed
  8bd7e92

  Copy full SHA for 8bd7e92

• Catch parsing raw resource

  JCZuurmond

  committed
  3344344

  Copy full SHA for 3344344

• Wait for an access connector

  JCZuurmond

  committed
  2938296

  Copy full SHA for 2938296

• Update mock prompts to not confirm creating an access connector for each storage account

  JCZuurmond

  committed
  fb72bfc

  Copy full SHA for fb72bfc

• Add mock prompt in integration test

  JCZuurmond

  committed
  86e0a30

  Copy full SHA for 86e0a30

• Add integration test for creating access connectors

  JCZuurmond

  committed
  1df318b

  Copy full SHA for 1df318b

Commits on Apr 16, 2024

• Rename ServicePrincipal migration methods

  JCZuurmond

  committed
  f67cfd5

  Copy full SHA for f67cfd5

• Return access connectors

  JCZuurmond

  committed
  c7dbd4d

  Copy full SHA for c7dbd4d

• Fix wrongly named argument

  JCZuurmond

  committed
  5b5fddc

  Copy full SHA for 5b5fddc

• Mock resources for access connector creation

  JCZuurmond

  committed
  2058462

  Copy full SHA for 2058462

• Move logging around execution result to run method

  JCZuurmond

  committed
  04ccf42

  Copy full SHA for 04ccf42

• Create storage credentials for access connectors

  JCZuurmond

  committed
  ed14bd9

  Copy full SHA for ed14bd9

• Add role name to StoragePermissionMapping

  JCZuurmond

  committed
  5eaea51

  Copy full SHA for 5eaea51

• Move prompts into run

  JCZuurmond

  committed
  4819369

  Copy full SHA for 4819369

• Move service principal migration info out of migrate service principals

  JCZuurmond

  committed
  9fb85bc

  Copy full SHA for 9fb85bc

• Expect service principal migration info in create access connectors

  JCZuurmond

  committed
  e09dcc2

  Copy full SHA for e09dcc2

• Add storage account to StoragePermissionMapping

  JCZuurmond

  committed
  66e4bde

  Copy full SHA for 66e4bde

• Build mapping with storage account permission

  JCZuurmond

  committed
  59b7117

  Copy full SHA for 59b7117

• Add hierarchy to levels

  JCZuurmond

  committed
  64d7812

  Copy full SHA for 64d7812

• Make role name an argument for creating an access connector

  JCZuurmond

  committed
  b2a7150

  Copy full SHA for b2a7150

• Expect storage account permissions mapping

  JCZuurmond

  committed
  177424d

  Copy full SHA for 177424d

• Build storage account permission mapping

  JCZuurmond

  committed
  862ad77

  Copy full SHA for 862ad77

• Pass role name up when creating access connectors

  JCZuurmond

  committed
  548f999

  Copy full SHA for 548f999

• Add note about validating storage credential

  JCZuurmond

  committed
  e285d84

  Copy full SHA for e285d84

• Format

  JCZuurmond

  committed
  9ce86f0

  Copy full SHA for 9ce86f0

• Return empty list

  JCZuurmond

  committed
  2a75fc0

  Copy full SHA for 2a75fc0

• Add missing type hints

  JCZuurmond

  committed
  f49a34f

  Copy full SHA for f49a34f

• Shorten service_principal_migration_info

  JCZuurmond

  committed
  ff607d4

  Copy full SHA for ff607d4

• Fix missing StoragePermissionMapping args

  JCZuurmond

  committed
  e42b375

  Copy full SHA for e42b375

• Shorten variable names

  JCZuurmond

  committed
  651eff2

  Copy full SHA for 651eff2

• Update test to include the storage account and role name in StoragePermissionMapping

  JCZuurmond

  committed
  2c0507d

  Copy full SHA for 2c0507d