fix(udf): try abs path after dlopen failed and more error catch #3359

vagetablechicken · 2023-07-12T10:08:11Z

No description provided.

tobegit3hub

LGTM

github-actions · 2023-07-12T10:50:06Z

SDK Test Report

  98 files ±0   98 suites ±0 1m 56s ⏱️ -20s
221 tests ±0 208 ✔️ ±0 13 💤 ±0 0 ❌ ±0
268 runs ±0 255 ✔️ ±0 13 💤 ±0 0 ❌ ±0

Results for commit 4af6b14. ± Comparison against base commit 5ad050f.

This pull request removes 30 and adds 9 tests. Note that renamed tests count towards both.

  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
 ) limit 10;](2)
 ) limit 10;](3)
 FROM db1.t1
 FROM t1
 WINDOW w1 AS (
 last join db2.t2 order by db2.t2.col1
…

com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[db1,  SELECT sum(t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlLastJoinWithMultipleDB[null,  SELECT sum(db1.t1.col1) over w1 as sum_t1_col1, db2.t2.str1 as t2_str1
 FROM db1.t1
 last join db2.t2 order by db2.t2.col1
 on db1.t1.col1 = db2.t2.col1 and db1.t1.col2 = db2.t2.col0
 WINDOW w1 AS (
  PARTITION BY db1.t1.col2 ORDER BY db1.t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](4)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db1.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: db1.t2.str1](2)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.col1](3)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[db1, SELECT t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Column Not found: .t2.str1](1)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlMultipleDBErrorTest[null, SELECT db2.t2.str1 as t2_str1
 FROM t1
 last join db2.t2 order by db2.t2.col1
 on t1.col1 = db2.t2.col1 and t1.col2 = db2.t2.col0;
, SQL parse error: Fail to transform data provider op: table t1 not exists in database []](5)
com._4paradigm.hybridse.sdk.SqlEngineTest ‑ sqlWindowLastJoin[ SELECT sum(t1.col1) over w1 as sum_t1_col1, t2.str1 as t2_str1
 FROM t1
 last join t2 order by t2.col1
 on t1.col1 = t2.col1 and t1.col2 = t2.col0
 WINDOW w1 AS (
  PARTITION BY t1.col2 ORDER BY t1.col1
  ROWS_RANGE BETWEEN 3 PRECEDING AND CURRENT ROW
 ) limit 10;](1)

♻️ This comment has been updated with latest results.

codecov · 2023-07-12T10:51:05Z

Codecov Report

Patch coverage: 23.07% and project coverage change: -0.03 ⚠️

Comparison is base (5ad050f) 75.65% compared to head (4af6b14) 75.62%.

Additional details and impacted files

@@             Coverage Diff              @@
##               main    #3359      +/-   ##
============================================
- Coverage     75.65%   75.62%   -0.03%     
  Complexity      414      414              
============================================
  Files           690      692       +2     
  Lines        125894   126083     +189     
  Branches       1206     1206              
============================================
+ Hits          95242    95354     +112     
- Misses        30427    30488      +61     
- Partials        225      241      +16

Impacted Files	Coverage Δ
hybridse/src/udf/dynamic_lib_manager.cc	`68.65% <23.07%> (-11.01%)`	⬇️

... and 17 files with indirect coverage changes

☔ View full report in Codecov by Sentry.
📢 Do you have feedback about the report comment? Let us know in this issue.

github-actions · 2023-07-12T10:55:12Z

HybridSE Mac Test Report

19 676 tests - 30 19 674 ✔️ - 30 9m 47s ⏱️ -33s
    245 suites -   3         2 💤 ±  0
      66 files -   1         0 ❌ ±  0

Results for commit 4af6b14. ± Comparison against base commit 5ad050f.

This pull request removes 30 tests.

DBMSSdkTest ‑ DatabasesAPITest
DBMSSdkTest ‑ ExecuteScriptAPITest
DBMSSdkTest ‑ ExecuteSqlTest
DBMSSdkTest ‑ GetInputSchema1
DBMSSdkTest ‑ GetInputSchema_ns_not_exist
DBMSSdkTest ‑ GetInputSchema_table_not_exist
DBMSSdkTest ‑ TableAPITest
DBMSSdkTest ‑ request_mode
SdkMultiInsert/DBMSSdkTest ‑ ExecuteQueryTest/0
SdkMultiInsert/DBMSSdkTest ‑ ExecuteQueryTest/1
…

github-actions · 2023-07-12T11:08:47Z

HybridSE Linux Test Report

19 706 tests ±0 19 704 ✔️ ±0 7m 25s ⏱️ - 2m 2s
    248 suites ±0         2 💤 ±0
      67 files ±0         0 ❌ ±0

Results for commit 4af6b14. ± Comparison against base commit 5ad050f.

github-actions · 2023-07-12T11:34:04Z

Linux Test Report

      57 files ±0     234 suites ±0 1h 10m 44s ⏱️ - 5m 11s
12 069 tests ±0 12 062 ✔️ ±0 7 💤 ±0 0 ❌ ±0
17 131 runs ±0 17 124 ✔️ ±0 7 💤 ±0 0 ❌ ±0

Results for commit 4af6b14. ± Comparison against base commit 5ad050f.

hybridse/src/udf/dynamic_lib_manager.cc

aceforeverd · 2023-07-12T16:04:20Z

hybridse/src/udf/dynamic_lib_manager.cc

+            err_msg = "can not open the dynamic library: " + file + ", error: " + dlerror() + ", try to use abs path";
+            LOG(WARNING) << err_msg;
+            // try to use abs path to avoid dlopen failed but it only works in the same path(e.g. in yarn mode)
+            char abs_path_buff[PATH_MAX];


reset errno to 0 here

https://linux.die.net/man/3/realpath
`If there is no error, realpath() returns a pointer to the resolved_path.

Otherwise, it returns a NULL pointer, the contents of the array resolved_path are undefined, and errno is set to indicate the error.`

set 0 before realpath is meaningless

fix(udf): try abs path after dlopen failed

4af6b14

github-actions bot added the execute-engine hybridse sql engine label Jul 12, 2023

tobegit3hub approved these changes Jul 12, 2023

View reviewed changes

vagetablechicken changed the title ~~fix(udf): try abs path after dlopen failed~~ fix(udf): try abs path after dlopen failed and more error catch Jul 12, 2023

aceforeverd reviewed Jul 12, 2023

View reviewed changes

aceforeverd approved these changes Jul 13, 2023

View reviewed changes

vagetablechicken merged commit 244c206 into main Jul 13, 2023

vagetablechicken deleted the vagetablechicken-patch-1 branch July 13, 2023 02:54

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(udf): try abs path after dlopen failed and more error catch #3359

fix(udf): try abs path after dlopen failed and more error catch #3359

vagetablechicken commented Jul 12, 2023

tobegit3hub left a comment

github-actions bot commented Jul 12, 2023 •

edited

Loading

codecov bot commented Jul 12, 2023 •

edited

Loading

github-actions bot commented Jul 12, 2023

github-actions bot commented Jul 12, 2023

github-actions bot commented Jul 12, 2023

aceforeverd Jul 12, 2023

vagetablechicken Jul 13, 2023

fix(udf): try abs path after dlopen failed and more error catch #3359

fix(udf): try abs path after dlopen failed and more error catch #3359

Conversation

vagetablechicken commented Jul 12, 2023

tobegit3hub left a comment

Choose a reason for hiding this comment

github-actions bot commented Jul 12, 2023 • edited Loading

SDK Test Report

codecov bot commented Jul 12, 2023 • edited Loading

Codecov Report

github-actions bot commented Jul 12, 2023

HybridSE Mac Test Report

github-actions bot commented Jul 12, 2023

HybridSE Linux Test Report

github-actions bot commented Jul 12, 2023

Linux Test Report

aceforeverd Jul 12, 2023

Choose a reason for hiding this comment

vagetablechicken Jul 13, 2023

Choose a reason for hiding this comment

github-actions bot commented Jul 12, 2023 •

edited

Loading

codecov bot commented Jul 12, 2023 •

edited

Loading