Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: add event_logger to test_connection and create_database commands #13468

Merged
merged 48 commits into from
Mar 9, 2021

Conversation

hughhhh
Copy link
Member

@hughhhh hughhhh commented Mar 4, 2021

SUMMARY

Leverage refactored event_logger with context management

Dependent on #13441

BEFORE/AFTER SCREENSHOTS OR ANIMATED GIF

TEST PLAN

ADDITIONAL INFORMATION

  • Has associated issue:
  • Changes UI
  • Requires DB Migration.
  • Confirm DB Migration upgrade and downgrade tested.
  • Introduces new feature or API
  • Removes existing feature or API

@hughhhh hughhhh requested a review from betodealmeida March 5, 2021 19:31
@pull-request-size pull-request-size bot added size/L and removed size/M labels Mar 7, 2021
Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I left a few comments and mentioned a couple nits, but I have one concern with the rollback() that's being called inside the event_logger context manager. We should move it out and call it before logging, to ensure that it goes through even if logging fails.

Comment on lines 54 to 59
with event_logger(
action="db_connection_failed",
engine=database.db_engine_spec.__name__,
):
db.session.rollback()
raise DatabaseConnectionFailedError()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

For places where we do a rollback we should it outside the context manager, otherwise if the logging fails the transaction won't be rolled back:

Suggested change
with event_logger(
action="db_connection_failed",
engine=database.db_engine_spec.__name__,
):
db.session.rollback()
raise DatabaseConnectionFailedError()
db.session.rollback()
event_logger.log_with_context(
action="db_connection_failed",
engine=database.db_engine_spec.__name__,
)
raise DatabaseConnectionFailedError()

Reversing the order ensures that even if the logging fails the rollback will still go through.

Comment on lines 70 to 74
with event_logger(
action=f"db_creation_failed.{ex.__class__.__name__}",
engine=database.db_engine_spec.__name__,
):
raise DatabaseCreateFailedError()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Total nit, but in places where we just raise an exception inside the context manager like here and in lines 94-97 I think it would make more sense to use the log_with_context function instead:

Suggested change
with event_logger(
action=f"db_creation_failed.{ex.__class__.__name__}",
engine=database.db_engine_spec.__name__,
):
raise DatabaseCreateFailedError()
event_logger.log_with_context(
action=f"db_creation_failed.{ex.__class__.__name__}",
engine=database.db_engine_spec.__name__,
)
raise DatabaseCreateFailedError()

Comment on lines 59 to 60
if database is None:
raise DBAPIError("Database is not found", None, None)
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We can actually remove this and fix the type annotation in DatabaseDAO.build_db_for_connection_test, the class method never returns None.

def test_connection_db_exception(
self, mock_event_logger, mock_build_db_for_connection_test
):
"""Test that users can't export databases they don't have access to"""
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Update docstring here and in the other 2 tests 😄

Comment on lines 542 to 543
with self.assertRaises(DatabaseTestConnectionUnexpectedError):
command_without_db_name.run()
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Now that we're using pytest, it's better to run tests using plain asserts instead, since it gives more context when the test fails. Also, we can check that the exception raised has the right error message:

Suggested change
with self.assertRaises(DatabaseTestConnectionUnexpectedError):
command_without_db_name.run()
with pytest.raises(DatabaseTestConnectionUnexpectedError) as excinfo:
command_without_db_name.run()
assert str(excinfo.value) == (
"Unexpected error occurred, please check your logs for details"
)

Same in the other 2 tests below.

@hughhhh hughhhh requested a review from betodealmeida March 8, 2021 19:29
Copy link
Member

@betodealmeida betodealmeida left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logger.exception(ex.exception)
event_logger.log_with_context(
action=f"db_creation_failed.{ex.__class__.__name__}",
engine=database.db_engine_spec.__name__,
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@hughhhh what did you want to do about making the engine string consistent around each of these logs so that we can group by the same engine later when observing the logs?

Copy link
Member Author

@hughhhh hughhhh Mar 9, 2021

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I ended up changing all the engine references in the event_logger.log_with_context to pull data from the database.db_engine_spec.__name__ I say we go with this for now and see the data then see if we need to create some mapping either in the service or downstream

@hughhhh hughhhh merged commit c91c455 into master Mar 9, 2021
@hughhhh hughhhh deleted the hugh/use-event-db branch March 9, 2021 13:17
allanco91 pushed a commit to allanco91/superset that referenced this pull request May 21, 2021
@mistercrunch mistercrunch added 🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels 🚢 1.2.0 labels Mar 12, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
🏷️ bot A label used by `supersetbot` to keep track of which PR where auto-tagged with release labels size/L 🚢 1.2.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants