Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

PythonMutator: replace stdin/stdout with files #1512

Merged
merged 2 commits into from
Jun 24, 2024

Conversation

kanterov
Copy link
Collaborator

Changes

Replace stdin/stdout with files in PythonMutator. Files are created in a temporary directory.

Rename ApplyPythonMutator to PythonMutator.

Add test for dyn.Location behavior during the "load" stage.

Tests

Unit tests

@kanterov kanterov requested a review from pietern June 20, 2024 20:02
Copy link
Contributor

@pietern pietern left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Comments are not blocking.


defer func() {
_ = outputFile.Close()
}()
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

You can do:

defer outputFile.Close()

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This will complain that we silently swallow the error, that's why I added a block

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

What complains? We use this same pattern elsewhere, AFAIK.

Copy link
Collaborator Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

IntelliJ :)

I guess you use VSCode that doesn't have this static check

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It also doesn't show up with staticcheck or Go itself.

@kanterov
Copy link
Collaborator Author

@pietern thanks for taking a look; I've replied in comment threads. I can change the defer block if you think we should despite swallowing the error. Otherwise, can you please add it to the merge queue?

@pietern pietern added this pull request to the merge queue Jun 24, 2024
Merged via the queue into databricks:main with commit 5ff0657 Jun 24, 2024
5 checks passed
pietern added a commit that referenced this pull request Jun 26, 2024
CLI:
 * Add link to documentation for Homebrew installation to README ([#1505](#1505)).
 * Fix `databricks configure` to use `DATABRICKS_CONFIG_FILE` environment variable if exists as config file ([#1325](#1325)).

Bundles:

The Terraform upgrade to v1.48.0 includes a fix for library order not being respected.

 * Fix conditional in query in `default-sql` template ([#1479](#1479)).
 * Remove user credentials specified in the Git origin URL ([#1494](#1494)).
 * Serialize dynamic value for `bundle validate` output ([#1499](#1499)).
 * Override variables with lookup value even if values has default value set ([#1504](#1504)).
 * Pause quality monitors when "mode: development" is used ([#1481](#1481)).
 * Return `fs.ModeDir` for Git folders in the workspace ([#1521](#1521)).
 * Upgrade TF provider to 1.48.0 ([#1527](#1527)).
 * Added support for complex variables ([#1467](#1467)).

Internal:
 * Add randIntn function ([#1475](#1475)).
 * Avoid multiple file tree traversals on bundle deploy ([#1493](#1493)).
 * Clean up unused code ([#1502](#1502)).
 * Use `dyn.InvalidValue` to indicate absence ([#1507](#1507)).
 * Add ApplyPythonMutator ([#1430](#1430)).
 * Set bool pointer to disable lock ([#1516](#1516)).
 * Allow the any type to be set to nil in `convert.FromTyped` ([#1518](#1518)).
 * Properly deal with nil values in `convert.FromTyped` ([#1511](#1511)).
 * Return `dyn.InvalidValue` instead of `dyn.NilValue` when errors happen ([#1514](#1514)).
 * PythonMutator: replace stdin/stdout with files ([#1512](#1512)).
 * Add context type and value to path rewriting ([#1525](#1525)).

API Changes:
 * Added schedule CRUD commands to `databricks lakeview`.
 * Added subscription CRUD commands to `databricks lakeview`.
 * Added `databricks apps start` command.

OpenAPI commit 7437dabb9dadee402c1fc060df4c1ce8cc5369f0 (2024-06-24)

Dependency updates:
 * Bump golang.org/x/text from 0.15.0 to 0.16.0 ([#1482](#1482)).
 * Bump golang.org/x/term from 0.20.0 to 0.21.0 ([#1483](#1483)).
 * Bump golang.org/x/mod from 0.17.0 to 0.18.0 ([#1484](#1484)).
 * Bump golang.org/x/oauth2 from 0.20.0 to 0.21.0 ([#1485](#1485)).
 * Bump github.com/briandowns/spinner from 1.23.0 to 1.23.1 ([#1495](#1495)).
 * Bump github.com/spf13/cobra from 1.8.0 to 1.8.1 ([#1496](#1496)).
 * Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0 ([#1522](#1522)).
@pietern pietern mentioned this pull request Jun 26, 2024
github-merge-queue bot pushed a commit that referenced this pull request Jun 26, 2024
CLI:
* Add link to documentation for Homebrew installation to README
([#1505](#1505)).
* Fix `databricks configure` to use `DATABRICKS_CONFIG_FILE` environment
variable if exists as config file
([#1325](#1325)).

Bundles:

The Terraform upgrade to v1.48.0 includes a fix for library order not
being respected.

* Fix conditional in query in `default-sql` template
([#1479](#1479)).
* Remove user credentials specified in the Git origin URL
([#1494](#1494)).
* Serialize dynamic value for `bundle validate` output
([#1499](#1499)).
* Override variables with lookup value even if values has default value
set ([#1504](#1504)).
* Pause quality monitors when "mode: development" is used
([#1481](#1481)).
* Return `fs.ModeDir` for Git folders in the workspace
([#1521](#1521)).
* Upgrade TF provider to 1.48.0
([#1527](#1527)).
* Added support for complex variables
([#1467](#1467)).

Internal:
* Add randIntn function
([#1475](#1475)).
* Avoid multiple file tree traversals on bundle deploy
([#1493](#1493)).
* Clean up unused code
([#1502](#1502)).
* Use `dyn.InvalidValue` to indicate absence
([#1507](#1507)).
* Add ApplyPythonMutator
([#1430](#1430)).
* Set bool pointer to disable lock
([#1516](#1516)).
* Allow the any type to be set to nil in `convert.FromTyped`
([#1518](#1518)).
* Properly deal with nil values in `convert.FromTyped`
([#1511](#1511)).
* Return `dyn.InvalidValue` instead of `dyn.NilValue` when errors happen
([#1514](#1514)).
* PythonMutator: replace stdin/stdout with files
([#1512](#1512)).
* Add context type and value to path rewriting
([#1525](#1525)).

API Changes:
 * Added schedule CRUD commands to `databricks lakeview`.
 * Added subscription CRUD commands to `databricks lakeview`.
 * Added `databricks apps start` command.

OpenAPI commit 7437dabb9dadee402c1fc060df4c1ce8cc5369f0 (2024-06-24)

Dependency updates:
* Bump golang.org/x/text from 0.15.0 to 0.16.0
([#1482](#1482)).
* Bump golang.org/x/term from 0.20.0 to 0.21.0
([#1483](#1483)).
* Bump golang.org/x/mod from 0.17.0 to 0.18.0
([#1484](#1484)).
* Bump golang.org/x/oauth2 from 0.20.0 to 0.21.0
([#1485](#1485)).
* Bump github.com/briandowns/spinner from 1.23.0 to 1.23.1
([#1495](#1495)).
* Bump github.com/spf13/cobra from 1.8.0 to 1.8.1
([#1496](#1496)).
* Bump github.com/databricks/databricks-sdk-go from 0.42.0 to 0.43.0
([#1522](#1522)).
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants