Hi Team, we are using UCX to assess workspace, job is running for 6 hrs and getting failed in craw_ table step and as eventual steps are dependent on tables remaining job getting failed #1397
-
|
Please find below error he spark driver has stopped unexpectedly and is restarting. Your notebook will be automatically reattached. Have increased number of workers, but still same issue |
Beta Was this translation helpful? Give feedback.
Replies: 1 comment 1 reply
-
|
Increase the size of the memory by editing the cluster policy. External metastore might be another slowdown. This task usually runs few minutes for 100k tables Soon we'll use beefier memory nodes by default. |
Beta Was this translation helpful? Give feedback.
-
|
@nfx i tired with 128 cores, seems still crawl_tables is not able to pull data, when I check in ucx schema and when I go to tables, not even single entry is being populated. Do we have any issue with crawl_tables notebook, as error log wise also it's not throwing anything |
Beta Was this translation helpful? Give feedback.
Increase the size of the memory by editing the cluster policy. External metastore might be another slowdown. This task usually runs few minutes for 100k tables
Soon we'll use beefier memory nodes by default.