Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[feat](nereids) simplify comparison for cast from smaller int type to bigger int type #48240

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

yujun777
Copy link
Collaborator

@yujun777 yujun777 commented Feb 24, 2025

What problem does this PR solve?

simplify comparison for cast from smaller int type to bigger int type.

for example:

suppose a is integer type, for expression a > 500000 + 100000, because 500000 and 100000 are int type, int + int is big int type, so the right side is big int type, then will rewrite as cast(a as bigint) > bigint(500000 + 100000), after fold constant, will got cast(a as bigint) > bigint(600000), but since 600000 can represent as a int type, we can simplify this predicate as a > int(600000)

Check List (For Author)

  • Test

    • Regression test
    • Unit Test
    • Manual test (add detailed scripts or steps below)
    • No need to test or manual test. Explain why:
      • This is a refactor/code format and no logic has been changed.
      • Previous test can cover this change.
      • No code files have been changed.
      • Other reason
  • Behavior changed:

    • No.
    • Yes.
  • Does this need documentation?

    • No.
    • Yes.

Check List (For Reviewer who merge this PR)

  • Confirm the release note
  • Confirm test cases
  • Confirm document
  • Add branch pick label

@hello-stephen
Copy link
Contributor

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR.

Please clearly describe your PR:

  1. What problem was fixed (it's best to include specific error reporting information). How it was fixed.
  2. Which behaviors were modified. What was the previous behavior, what is it now, why was it modified, and what possible impacts might there be.
  3. What features were added. Why was this function added?
  4. Which code was refactored and why was this part of the code refactored?
  5. Which functions were optimized and what is the difference before and after the optimization?

@yujun777
Copy link
Collaborator Author

run buildall

@yujun777 yujun777 force-pushed the feat-simplify-int-compare branch from 40e9006 to 7c7dd2a Compare February 24, 2025 09:41
@yujun777
Copy link
Collaborator Author

run buildall

@yujun777 yujun777 changed the title [feat](nereids) simplify comparison from cast from smaller int type to bigger int type [feat](nereids) simplify comparison for cast from smaller int type to bigger int type Feb 24, 2025
@doris-robot
Copy link

TPC-H: Total hot run time: 31644 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 7c7dd2a2ac255a5dc7b56a9f41d019fe2fae32f2, data reload: false

------ Round 1 ----------------------------------
q1	11904	5015	5095	5015
q2	999	297	171	171
q3	879	1256	708	708
q4	711	1026	519	519
q5	2123	2320	2364	2320
q6	195	168	134	134
q7	914	809	686	686
q8	1114	1311	1208	1208
q9	4781	4676	4679	4676
q10	2255	2339	1875	1875
q11	426	275	263	263
q12	309	353	222	222
q13	3361	3641	3033	3033
q14	233	225	209	209
q15	517	485	469	469
q16	623	600	575	575
q17	578	879	329	329
q18	6604	6235	6269	6235
q19	554	939	529	529
q20	315	326	195	195
q21	2795	2108	1967	1967
q22	371	342	306	306
Total cold run time: 42561 ms
Total hot run time: 31644 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5042	5088	5138	5088
q2	244	330	234	234
q3	2203	2676	2326	2326
q4	1438	1832	1392	1392
q5	4206	4110	4153	4110
q6	207	162	122	122
q7	1858	1841	1661	1661
q8	2533	2407	2412	2407
q9	6819	6880	6781	6781
q10	3113	3079	2626	2626
q11	560	497	470	470
q12	645	727	568	568
q13	3326	3630	3078	3078
q14	274	276	272	272
q15	488	454	456	454
q16	605	659	617	617
q17	1093	1543	1321	1321
q18	7197	6969	6898	6898
q19	751	787	824	787
q20	1916	1936	1790	1790
q21	5198	4708	4545	4545
q22	634	539	538	538
Total cold run time: 50350 ms
Total hot run time: 48085 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 183107 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 7c7dd2a2ac255a5dc7b56a9f41d019fe2fae32f2, data reload: false

query1	995	383	371	371
query2	6518	1827	1848	1827
query3	6805	217	209	209
query4	25985	23697	23548	23548
query5	5756	652	490	490
query6	330	198	175	175
query7	4593	518	301	301
query8	304	237	220	220
query9	8653	2565	2551	2551
query10	514	300	255	255
query11	15691	15054	14888	14888
query12	170	107	109	107
query13	1662	517	419	419
query14	10909	6033	6086	6033
query15	219	197	167	167
query16	7679	638	451	451
query17	1542	733	573	573
query18	1985	397	283	283
query19	245	174	151	151
query20	121	109	110	109
query21	207	119	99	99
query22	4148	4149	4141	4141
query23	33783	32747	32894	32747
query24	7972	2389	2394	2389
query25	502	457	410	410
query26	1232	276	152	152
query27	2681	492	338	338
query28	4309	2418	2382	2382
query29	676	540	411	411
query30	237	188	159	159
query31	917	846	781	781
query32	101	65	59	59
query33	559	377	297	297
query34	752	835	487	487
query35	806	810	719	719
query36	1014	990	880	880
query37	120	95	80	80
query38	4199	4175	4030	4030
query39	1453	1389	1393	1389
query40	206	117	103	103
query41	53	52	50	50
query42	118	108	104	104
query43	469	501	476	476
query44	1282	780	812	780
query45	177	171	161	161
query46	856	1018	654	654
query47	1729	1779	1709	1709
query48	391	421	302	302
query49	799	515	413	413
query50	685	749	408	408
query51	4174	4147	4066	4066
query52	109	110	98	98
query53	225	269	180	180
query54	491	478	423	423
query55	83	81	77	77
query56	257	266	255	255
query57	1124	1146	1065	1065
query58	277	250	256	250
query59	2671	2552	2512	2512
query60	296	286	271	271
query61	149	138	142	138
query62	786	744	652	652
query63	247	197	199	197
query64	4301	1097	760	760
query65	3295	3149	3111	3111
query66	1000	418	307	307
query67	15700	15508	15444	15444
query68	5178	785	514	514
query69	468	288	272	272
query70	1171	1152	1115	1115
query71	391	339	263	263
query72	5938	3600	3681	3600
query73	714	743	347	347
query74	8908	9069	8862	8862
query75	3130	3145	2709	2709
query76	3079	1164	735	735
query77	469	363	279	279
query78	10071	10192	9252	9252
query79	1372	887	596	596
query80	684	537	443	443
query81	508	277	241	241
query82	423	133	96	96
query83	174	173	154	154
query84	244	86	80	80
query85	754	358	297	297
query86	331	316	295	295
query87	4448	4500	4383	4383
query88	3029	2218	2212	2212
query89	396	308	280	280
query90	1855	193	195	193
query91	136	140	110	110
query92	58	59	57	57
query93	1161	1011	576	576
query94	608	411	363	363
query95	349	267	260	260
query96	490	531	272	272
query97	2736	2800	2746	2746
query98	228	204	202	202
query99	1277	1387	1259	1259
Total cold run time: 269072 ms
Total hot run time: 183107 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.62 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 7c7dd2a2ac255a5dc7b56a9f41d019fe2fae32f2, data reload: false

query1	0.04	0.04	0.06
query2	0.07	0.04	0.03
query3	0.23	0.07	0.06
query4	1.62	0.10	0.10
query5	0.42	0.42	0.39
query6	1.15	0.66	0.65
query7	0.03	0.02	0.02
query8	0.04	0.03	0.03
query9	0.60	0.51	0.52
query10	0.57	0.57	0.55
query11	0.15	0.10	0.10
query12	0.15	0.10	0.11
query13	0.61	0.59	0.61
query14	2.69	2.72	2.67
query15	0.92	0.87	0.85
query16	0.38	0.38	0.39
query17	1.06	1.03	1.02
query18	0.22	0.20	0.20
query19	1.92	1.85	1.95
query20	0.01	0.01	0.01
query21	15.35	0.87	0.53
query22	0.74	1.19	0.73
query23	14.89	1.36	0.59
query24	6.62	1.83	1.89
query25	0.53	0.24	0.12
query26	0.60	0.15	0.13
query27	0.06	0.05	0.05
query28	10.49	0.80	0.42
query29	12.57	3.96	3.29
query30	0.26	0.09	0.06
query31	2.86	0.59	0.37
query32	3.22	0.55	0.47
query33	2.95	2.96	3.05
query34	15.77	5.11	4.57
query35	4.55	4.54	4.53
query36	0.67	0.50	0.49
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.03	0.02	0.02
query40	0.17	0.13	0.14
query41	0.08	0.03	0.03
query42	0.04	0.03	0.02
query43	0.04	0.04	0.03
Total cold run time: 105.52 s
Total hot run time: 31.62 s

@yujun777
Copy link
Collaborator Author

run buildall

@doris-robot
Copy link

TPC-H: Total hot run time: 31302 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit e239811c772e58b8c45a34ba1ce5ed39ef839b39, data reload: false

------ Round 1 ----------------------------------
q1	17639	5286	5030	5030
q2	2054	295	168	168
q3	10532	1230	734	734
q4	10213	989	513	513
q5	7533	2452	2285	2285
q6	196	167	135	135
q7	891	750	594	594
q8	9294	1298	1104	1104
q9	4833	4627	4694	4627
q10	6799	2309	1879	1879
q11	489	274	247	247
q12	336	359	214	214
q13	17766	3678	3044	3044
q14	248	217	209	209
q15	513	483	465	465
q16	624	594	580	580
q17	557	851	325	325
q18	6915	6305	6243	6243
q19	1199	934	522	522
q20	310	318	182	182
q21	2684	2159	1907	1907
q22	366	332	295	295
Total cold run time: 101991 ms
Total hot run time: 31302 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5083	5044	5087	5044
q2	256	341	233	233
q3	2145	2645	2371	2371
q4	1402	1841	1412	1412
q5	4300	4202	4204	4202
q6	213	167	125	125
q7	1899	1906	1650	1650
q8	2580	2612	2550	2550
q9	7166	7170	7115	7115
q10	3043	3159	2768	2768
q11	562	533	498	498
q12	662	779	601	601
q13	3514	3830	3325	3325
q14	276	300	293	293
q15	509	484	458	458
q16	624	667	628	628
q17	1115	1593	1307	1307
q18	7599	7500	7378	7378
q19	762	775	812	775
q20	1965	2054	1869	1869
q21	5351	4970	4777	4777
q22	623	544	529	529
Total cold run time: 51649 ms
Total hot run time: 49908 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 190654 ms
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit e239811c772e58b8c45a34ba1ce5ed39ef839b39, data reload: false

query1	1316	983	949	949
query2	6236	1841	1800	1800
query3	10973	4302	4360	4302
query4	57096	25223	23369	23369
query5	5145	510	491	491
query6	360	210	198	198
query7	5135	506	312	312
query8	327	241	245	241
query9	6323	2750	2766	2750
query10	422	308	254	254
query11	15141	14974	14775	14775
query12	160	106	105	105
query13	1158	566	406	406
query14	10133	6255	6387	6255
query15	209	203	173	173
query16	7130	645	471	471
query17	1049	738	557	557
query18	1565	401	301	301
query19	214	194	166	166
query20	129	127	118	118
query21	215	127	101	101
query22	4722	4708	4849	4708
query23	34029	33263	33545	33263
query24	6463	2423	2471	2423
query25	535	489	435	435
query26	691	286	159	159
query27	2036	567	352	352
query28	3036	2528	2505	2505
query29	569	547	417	417
query30	216	186	155	155
query31	888	850	770	770
query32	69	64	62	62
query33	458	407	307	307
query34	771	861	547	547
query35	849	833	735	735
query36	966	989	898	898
query37	119	108	71	71
query38	4251	4303	4175	4175
query39	1477	1485	1436	1436
query40	212	115	100	100
query41	50	50	51	50
query42	133	114	110	110
query43	486	504	479	479
query44	1349	843	861	843
query45	177	176	169	169
query46	911	1048	676	676
query47	1848	1926	1848	1848
query48	426	424	344	344
query49	724	519	425	425
query50	698	775	422	422
query51	4366	4229	4247	4229
query52	114	106	98	98
query53	228	262	191	191
query54	495	504	415	415
query55	88	86	85	85
query56	280	269	255	255
query57	1186	1201	1131	1131
query58	249	247	257	247
query59	2777	2967	2850	2850
query60	309	281	272	272
query61	115	135	148	135
query62	731	742	672	672
query63	230	193	202	193
query64	1452	1001	642	642
query65	3198	3139	3096	3096
query66	694	393	322	322
query67	16029	15487	15328	15328
query68	4324	800	541	541
query69	508	364	266	266
query70	1170	1157	1122	1122
query71	429	293	269	269
query72	6266	3711	3671	3671
query73	788	747	360	360
query74	9129	9141	9022	9022
query75	3348	3129	2680	2680
query76	3397	1173	727	727
query77	553	350	291	291
query78	10178	10175	9338	9338
query79	1669	846	628	628
query80	786	537	453	453
query81	518	281	232	232
query82	424	129	99	99
query83	186	171	178	171
query84	281	95	72	72
query85	732	400	295	295
query86	334	304	292	292
query87	4579	4453	4372	4372
query88	3055	2398	2362	2362
query89	392	314	283	283
query90	1738	196	190	190
query91	134	139	120	120
query92	63	64	54	54
query93	1300	1012	576	576
query94	618	459	294	294
query95	351	267	248	248
query96	522	545	292	292
query97	2792	2859	2734	2734
query98	229	205	202	202
query99	1301	1348	1259	1259
Total cold run time: 294230 ms
Total hot run time: 190654 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.19 s
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit e239811c772e58b8c45a34ba1ce5ed39ef839b39, data reload: false

query1	0.03	0.03	0.04
query2	0.07	0.03	0.03
query3	0.24	0.07	0.06
query4	1.61	0.10	0.11
query5	0.41	0.40	0.40
query6	1.17	0.66	0.66
query7	0.02	0.02	0.02
query8	0.04	0.03	0.03
query9	0.59	0.52	0.50
query10	0.57	0.57	0.58
query11	0.16	0.10	0.11
query12	0.14	0.12	0.11
query13	0.61	0.60	0.61
query14	2.68	2.73	2.72
query15	0.93	0.86	0.87
query16	0.36	0.38	0.40
query17	1.08	1.02	1.03
query18	0.22	0.20	0.19
query19	1.87	1.77	1.96
query20	0.01	0.01	0.01
query21	15.36	0.89	0.55
query22	0.74	1.19	0.78
query23	14.83	1.39	0.66
query24	6.93	1.81	0.41
query25	0.37	0.29	0.14
query26	0.64	0.16	0.13
query27	0.05	0.05	0.05
query28	8.95	0.89	0.42
query29	12.55	3.94	3.29
query30	0.25	0.08	0.05
query31	2.84	0.59	0.38
query32	3.22	0.55	0.47
query33	2.95	3.04	2.96
query34	15.69	5.19	4.47
query35	4.52	4.51	4.48
query36	0.67	0.49	0.48
query37	0.09	0.06	0.06
query38	0.06	0.04	0.04
query39	0.02	0.02	0.02
query40	0.17	0.13	0.12
query41	0.07	0.03	0.02
query42	0.04	0.03	0.02
query43	0.04	0.03	0.03
Total cold run time: 103.86 s
Total hot run time: 30.19 s

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

3 participants