Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Error deploying task with library dependency in object storage #699

Closed
bradleyjamrozik opened this issue Aug 25, 2023 · 6 comments · Fixed by #702
Closed

Error deploying task with library dependency in object storage #699

bradleyjamrozik opened this issue Aug 25, 2023 · 6 comments · Fixed by #702
Assignees
Labels
Bug Something isn't working DABs DABs related issues

Comments

@bradleyjamrozik
Copy link

I receive an error when deploying a job task with a library dependency on >=v202.0. This error does not happen on v0.201.0.

time=2023-08-25T16:40:14.463-05:00 level=ERROR source=mutator.go:25 msg="Error: no library found for abfss://databricks@<storageaccount>.dfs.core.windows.net/libraries/ozspark/ozspark_databricks-0.0.1-py3-none-any.whl" mutator=seq mutator=deploy mutator=seq mutator=seq mutator=deferred time=2023-08-25T16:40:14.463-05:00 level=ERROR source=mutator.go:25 msg="Error: no library found for abfss://databricks@<storageaccount>.dfs.core.windows.net/libraries/ozspark/ozspark_databricks-0.0.1-py3-none-any.whl" mutator=seq mutator=deploy mutator=seq mutator=seq time=2023-08-25T16:40:14.463-05:00 level=ERROR source=mutator.go:25 msg="Error: no library found for abfss://databricks@<storageaccount>.dfs.core.windows.net/libraries/ozspark/ozspark_databricks-0.0.1-py3-none-any.whl" mutator=seq mutator=deploy mutator=seq time=2023-08-25T16:40:14.463-05:00 level=ERROR source=mutator.go:25 msg="Error: no library found for abfss://databricks@<storageaccount>.dfs.core.windows.net/libraries/ozspark/ozspark_databricks-0.0.1-py3-none-any.whl" mutator=seq mutator=deploy time=2023-08-25T16:40:14.464-05:00 level=ERROR source=mutator.go:25 msg="Error: no library found for abfss://databricks@<storageaccount>.dfs.core.windows.net/libraries/ozspark/ozspark_databricks-0.0.1-py3-none-any.whl" mutator=seq Error: no library found for abfss://databricks@<storageaccount>.dfs.core.windows.net/libraries/ozspark/ozspark_databricks-0.0.1-py3-none-any.whl

@bradleyjamrozik
Copy link
Author

Clarification: this is using databricks asset bundles

@andrewnester andrewnester added Bug Something isn't working DABs DABs related issues labels Aug 28, 2023
@andrewnester
Copy link
Contributor

Thanks for the report! This is indeed a bug on our side in how we detect if the library used is local or remote

func isLocalLibrary(library *compute.Library) bool {

@bradleyjamrozik
Copy link
Author

I upgraded to v0.203.2 and am still receiving the "no library found" error for the wheel located in object storage, while I do not get this error on v0.201.0. Is there something I need to be doing differently in my yaml file?
image

@andrewnester
Copy link
Contributor

The fix will only be available in the upcoming next release this week

@bradleyjamrozik
Copy link
Author

Ahhh ok. Understood, thank you!

@andrewnester
Copy link
Contributor

The fix has been just released in 0.203.3 version, feel free to try it out

lennartkats-db pushed a commit that referenced this issue Sep 1, 2023
## Changes
Fixes #699

## Tests
Added unit test
arpitjasa-db pushed a commit to arpitjasa-db/cli that referenced this issue Sep 7, 2023
## Changes
Fixes databricks#699

## Tests
Added unit test

Signed-off-by: Arpit Jasapara <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Something isn't working DABs DABs related issues
Projects
None yet
Development

Successfully merging a pull request may close this issue.

2 participants