Skip to content

Commit

Permalink
vdk-impala: add udf hive2server user error classification (#2056)
Browse files Browse the repository at this point in the history
what: added a new error to be classified as user exception in the
vdk-impala plugin

why: a use case came up where the following excerpt of an exception was
classified as platform error:

`File "/vdk/site-packages/impala/hiveserver2.py", line 1397, in fetch
    resp = self._rpc('FetchResults', req, False)
  File "/vdk/site-packages/impala/hiveserver2.py", line 1085, in _rpc
    err_if_rpc_not_ok(response)
File "/vdk/site-packages/impala/hiveserver2.py", line 781, in
err_if_rpc_not_ok
    raise HiveServer2Error(resp.status.errorMessage)
impala.error.HiveServer2Error: UDF ERROR: Decimal expression overflowed


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
File "/vdk/site-packages/vdk/internal/builtin_plugins/run/data_job.py",
line 73, in run_step
    step_executed = step.runner_func(step, context.job_input)`

testing: added case to the unit test

Signed-off-by: mrMoZ1 <[email protected]>
  • Loading branch information
Momchil Z authored May 12, 2023
1 parent 8470dd1 commit 6749601
Show file tree
Hide file tree
Showing 2 changed files with 16 additions and 0 deletions.
Original file line number Diff line number Diff line change
Expand Up @@ -20,6 +20,7 @@ def is_impala_user_error(received_exception):
or _is_quota_exceeded(received_exception)
or _is_other_query_error(received_exception)
or _is_property_unsupported_value_error(received_exception)
or _is_udf_hive_server2_error(received_exception)
)


Expand Down Expand Up @@ -119,3 +120,11 @@ def _is_quota_exceeded(exception):
classname_with_package="impala.error.OperationalError",
exception_message_matcher_regex=".*DiskSpace quota of .* is exceeded.*",
)


def _is_udf_hive_server2_error(exception):
return errors.exception_matches(
exception,
classname_with_package="impala.error.HiveServer2Error",
exception_message_matcher_regex=".*UDF ERROR:.*",
)
Original file line number Diff line number Diff line change
Expand Up @@ -92,3 +92,10 @@ def test_user_error_classification(self):
)
)
)
self.assertTrue(
is_impala_user_error(
HiveServer2Error(
"impala.error.HiveServer2Error: UDF ERROR: Decimal expression overflowed"
)
)
)

0 comments on commit 6749601

Please sign in to comment.