Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

coprocessor request with projection and order by takes too much memory #36041

Closed
tiancaiamao opened this issue Jul 8, 2022 · 0 comments · Fixed by #36047
Closed

coprocessor request with projection and order by takes too much memory #36041

tiancaiamao opened this issue Jul 8, 2022 · 0 comments · Fixed by #36047
Labels
type/enhancement The issue or PR belongs to an enhancement.

Comments

@tiancaiamao
Copy link
Contributor

Enhancement

With projection, no order by

time mysql -h 127.0.0.1 -u root -P 4000 -B ens_rb -e 'set @@tidb_enable_paging = 1; select  _tidb_rowid,  internal_key,  int_class,  next_cycle_date,  cycle_freq,  int_day,  last_cycle_date,  last_true_cycle_date,  int_type,  actual_rate,  float_rate,  spread_rate,  spread_percent,  acct_fixed_rate,  acct_spread_rate,  acct_percent_rate,  real_rate,  penalty_odi_rate_type,  int_accrued_ctd,  int_accrued,  int_accrued_prev,  int_adj_ctd,  int_adj,  int_adj_prev,  int_posted_ctd,  int_posted,  int_past_due,  last_int_past_due,  int_cap,  int_cap_amt,  last_accrual_date,  int_appl_type,  rate_effect_type,  roll_date,  roll_freq,  roll_day,  discnt_retain_int,  discnt_ui_flag,  ui_int,  ui_penalty_amt,  discnt_int,  discnt_int_prev,  calc_begin_date,  calc_end_date,  last_change_date,  last_change_user_id,  int_rem_days,  year_basis,  month_basis,  dac_value,  company,  min_int_rate,  max_int_rate,  tax_type,  tax_rate,  tax_accrued_ctd,  tax_accrued,  tax_posted_ctd,  tax_posted,  last_cycle_date_pre,  int_ind,  rate_change_ind,  accr_int_day,  accr_period_freq,  next_accr_date,  agg,  agree_agg,  agree_post,  int_accrued_calc_ctd,  int_accrued_diff,  int_accrued_t,  int_amt,  is_cycle,  is_retry,  last_roll_date,  tax_accrued_calc_ctd,  tax_accrued_diff,  int_calc_bal,  calc_by_int,  settle_cycle_date,  agree_change_type,  agree_fixed_rate,  agree_percent_rate,  agree_spread_rate,  td_int_num_days,  split_rate_flag,  int_tax_t,  acct_fixed_tax_rate,  acct_spread_tax_rate,  acct_percent_tax_rate,  td_accr_int_day,  agree_reduce_amt,  td_last_accr_date,  system_id,  float_type,  follow_int_day_type,  follow_trace_natural_days,  follow_trace_workday_days,  tran_timestamp,  client_no,  layer_agreement,  int_appl_type_prev, tax_accrued,  tax_posted_ctd,  tax_posted,  last_cycle_date_pre,  int_ind,  rate_change_ind,  accr_int_day,  accr_period_freq,  next_accr_date,  agg,  agree_agg,  agree_post,  int_accrued_calc_ctd,  int_accrued_diff,  int_accrued_t,  int_amt,  is_cycle,  is_retry,  last_roll_date,  tax_accrued_calc_ctd,  tax_accrued_diff,  int_calc_bal,  calc_by_int,  settle_cycle_date,  agree_change_type,  agree_fixed_rate,  agree_percent_rate,  agree_spread_rate,  td_int_num_days,  split_rate_flag,  int_tax_t,  acct_fixed_tax_rate,  acct_spread_tax_rate,  acct_percent_tax_rate,  td_accr_int_day,  agree_reduce_amt,  td_last_accr_date,  system_id,  float_type,  follow_int_day_type,  follow_trace_natural_days,  follow_trace_workday_days,  tran_timestamp,  client_no,  layer_agreement,  int_appl_type_prev FROM `ens_rb003`.`rb_acct_int_detail`' > /dev/null

real        2m31.599s
user        1m23.402s
sys        0m6.671s

With projection, and order by:

time mysql -h 127.0.0.1 -u root -P 4000 -B ens_rb -e 'set @@tidb_enable_paging = 1; select  _tidb_rowid,  internal_key,  int_class,  next_cycle_date,  cycle_freq,  int_day,  last_cycle_date,  last_true_cycle_date,  int_type,  actual_rate,  float_rate,  spread_rate,  spread_percent,  acct_fixed_rate,  acct_spread_rate,  acct_percent_rate,  real_rate,  penalty_odi_rate_type,  int_accrued_ctd,  int_accrued,  int_accrued_prev,  int_adj_ctd,  int_adj,  int_adj_prev,  int_posted_ctd,  int_posted,  int_past_due,  last_int_past_due,  int_cap,  int_cap_amt,  last_accrual_date,  int_appl_type,  rate_effect_type,  roll_date,  roll_freq,  roll_day,  discnt_retain_int,  discnt_ui_flag,  ui_int,  ui_penalty_amt,  discnt_int,  discnt_int_prev,  calc_begin_date,  calc_end_date,  last_change_date,  last_change_user_id,  int_rem_days,  year_basis,  month_basis,  dac_value,  company,  min_int_rate,  max_int_rate,  tax_type,  tax_rate,  tax_accrued_ctd,  tax_accrued,  tax_posted_ctd,  tax_posted,  last_cycle_date_pre,  int_ind,  rate_change_ind,  accr_int_day,  accr_period_freq,  next_accr_date,  agg,  agree_agg,  agree_post,  int_accrued_calc_ctd,  int_accrued_diff,  int_accrued_t,  int_amt,  is_cycle,  is_retry,  last_roll_date,  tax_accrued_calc_ctd,  tax_accrued_diff,  int_calc_bal,  calc_by_int,  settle_cycle_date,  agree_change_type,  agree_fixed_rate,  agree_percent_rate,  agree_spread_rate,  td_int_num_days,  split_rate_flag,  int_tax_t,  acct_fixed_tax_rate,  acct_spread_tax_rate,  acct_percent_tax_rate,  td_accr_int_day,  agree_reduce_amt,  td_last_accr_date,  system_id,  float_type,  follow_int_day_type,  follow_trace_natural_days,  follow_trace_workday_days,  tran_timestamp,  client_no,  layer_agreement,  int_appl_type_prev, tax_accrued,  tax_posted_ctd,  tax_posted,  last_cycle_date_pre,  int_ind,  rate_change_ind,  accr_int_day,  accr_period_freq,  next_accr_date,  agg,  agree_agg,  agree_post,  int_accrued_calc_ctd,  int_accrued_diff,  int_accrued_t,  int_amt,  is_cycle,  is_retry,  last_roll_date,  tax_accrued_calc_ctd,  tax_accrued_diff,  int_calc_bal,  calc_by_int,  settle_cycle_date,  agree_change_type,  agree_fixed_rate,  agree_percent_rate,  agree_spread_rate,  td_int_num_days,  split_rate_flag,  int_tax_t,  acct_fixed_tax_rate,  acct_spread_tax_rate,  acct_percent_tax_rate,  td_accr_int_day,  agree_reduce_amt,  td_last_accr_date,  system_id,  float_type,  follow_int_day_type,  follow_trace_natural_days,  follow_trace_workday_days,  tran_timestamp,  client_no,  layer_agreement,  int_appl_type_prev FROM `ens_rb003`.`rb_acct_int_detail` order by _tidb_rowid' > /dev/null

real        2m31.506s
user        1m25.170s
sys        0m6.520s

image

I mean, even I'm using coprocessor paging protocol,
even after the fix #35975

This case still cost way too much. There must be something wrong!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
type/enhancement The issue or PR belongs to an enhancement.
Projects
None yet
Development

Successfully merging a pull request may close this issue.

1 participant