Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[opt](routine-load) end Kafka consume when meets partition EOF #32046

Merged
merged 1 commit into from
Mar 12, 2024
Merged

[opt](routine-load) end Kafka consume when meets partition EOF #32046

merged 1 commit into from
Mar 12, 2024

Conversation

sollhui
Copy link
Contributor

@sollhui sollhui commented Mar 11, 2024

Proposed changes

When all the partitions held by a consumer reach EOF, the consumer can end and give up resources to avoid ineffective occupation of resources.

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@doris-robot
Copy link

Thank you for your contribution to Apache Doris.
Don't know what should be done next? See How to process your PR

@sollhui
Copy link
Contributor Author

sollhui commented Mar 11, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 38373 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 111487b349c835beb0618d26c76d1279513c098e, data reload: false

------ Round 1 ----------------------------------
q1	19086	4145	4215	4145
q2	2291	149	150	149
q3	11633	973	957	957
q4	5145	953	963	953
q5	8091	2965	3080	2965
q6	199	129	131	129
q7	1353	888	820	820
q8	9500	2118	2092	2092
q9	7307	6535	6492	6492
q10	8224	2561	2548	2548
q11	417	220	217	217
q12	794	324	329	324
q13	17974	2994	2976	2976
q14	275	244	261	244
q15	505	469	459	459
q16	477	397	402	397
q17	955	880	857	857
q18	6814	5864	5914	5864
q19	1575	1526	1530	1526
q20	546	284	275	275
q21	7612	3683	3689	3683
q22	792	301	317	301
Total cold run time: 111565 ms
Total hot run time: 38373 ms

----- Round 2, with runtime_filter_mode=off -----
q1	3985	3986	3966	3966
q2	320	216	225	216
q3	2963	2928	2959	2928
q4	1843	1829	1817	1817
q5	5244	5232	5243	5232
q6	204	115	115	115
q7	2272	1818	1850	1818
q8	3216	3274	3281	3274
q9	8492	8528	8555	8528
q10	6126	3719	3726	3719
q11	546	443	456	443
q12	721	561	562	561
q13	7396	2839	2834	2834
q14	276	247	254	247
q15	488	452	437	437
q16	449	412	401	401
q17	1695	1640	1659	1640
q18	7876	7460	7318	7318
q19	1623	1604	1629	1604
q20	1893	1695	1701	1695
q21	4956	4875	4900	4875
q22	512	447	431	431
Total cold run time: 63096 ms
Total hot run time: 54099 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 35.00% (8556/24446)
Line Coverage: 26.83% (69532/259145)
Region Coverage: 26.08% (36082/138339)
Branch Coverage: 23.05% (18444/80034)
Coverage Report: http://coverage.selectdb-in.cc/coverage/111487b349c835beb0618d26c76d1279513c098e_111487b349c835beb0618d26c76d1279513c098e/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 185899 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 111487b349c835beb0618d26c76d1279513c098e, data reload: false

query1	930	365	346	346
query2	7408	1965	2001	1965
query3	6712	213	208	208
query4	24954	20880	20886	20880
query5	4324	531	492	492
query6	257	189	193	189
query7	4633	312	293	293
query8	233	161	171	161
query9	8447	2157	2179	2157
query10	467	231	252	231
query11	15207	14553	14564	14553
query12	136	90	112	90
query13	1653	431	434	431
query14	10583	10391	10066	10066
query15	239	193	196	193
query16	7781	279	272	272
query17	1863	592	560	560
query18	1955	285	259	259
query19	191	155	150	150
query20	92	88	86	86
query21	195	122	124	122
query22	4610	4452	4479	4452
query23	31387	30892	30671	30671
query24	11170	3077	3022	3022
query25	632	390	373	373
query26	1284	155	150	150
query27	2974	347	353	347
query28	6580	1818	1788	1788
query29	1029	600	593	593
query30	304	147	148	147
query31	938	730	738	730
query32	104	60	57	57
query33	752	252	254	252
query34	1035	462	482	462
query35	847	661	628	628
query36	913	809	827	809
query37	140	78	79	78
query38	3273	3162	3161	3161
query39	1413	1366	1367	1366
query40	208	113	120	113
query41	71	64	64	64
query42	107	113	105	105
query43	424	405	384	384
query44	1107	698	708	698
query45	271	261	264	261
query46	1050	773	772	772
query47	1635	1561	1568	1561
query48	422	343	341	341
query49	1141	338	334	334
query50	784	380	381	380
query51	6750	6586	6547	6547
query52	107	102	94	94
query53	349	285	283	283
query54	318	243	246	243
query55	87	84	82	82
query56	240	233	222	222
query57	1094	1007	981	981
query58	255	221	218	218
query59	2489	2182	2265	2182
query60	261	258	258	258
query61	127	110	112	110
query62	646	407	403	403
query63	302	283	279	279
query64	5297	3406	3196	3196
query65	3068	3023	2964	2964
query66	1420	331	349	331
query67	15056	14376	14430	14376
query68	6010	555	563	555
query69	603	380	380	380
query70	1207	1117	1113	1113
query71	483	288	275	275
query72	6542	2658	2466	2466
query73	742	316	312	312
query74	6773	6529	6503	6503
query75	9038	8033	8071	8033
query76	4211	1083	1168	1083
query77	619	248	255	248
query78	10195	9645	9548	9548
query79	8172	497	506	497
query80	1777	383	401	383
query81	551	206	208	206
query82	1409	201	197	197
query83	310	144	148	144
query84	286	78	76	76
query85	1649	331	315	315
query86	475	298	280	280
query87	3423	3248	3277	3248
query88	4246	2282	2252	2252
query89	556	357	353	353
query90	2001	176	170	170
query91	158	129	125	125
query92	54	51	50	50
query93	6378	525	524	524
query94	1199	183	190	183
query95	441	351	342	342
query96	592	267	257	257
query97	4428	4250	4262	4250
query98	247	210	212	210
query99	1213	749	764	749
Total cold run time: 298719 ms
Total hot run time: 185899 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.63 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 111487b349c835beb0618d26c76d1279513c098e, data reload: false

query1	0.04	0.03	0.03
query2	0.05	0.02	0.03
query3	0.23	0.06	0.06
query4	1.67	0.10	0.09
query5	0.48	0.48	0.52
query6	1.26	0.68	0.68
query7	0.02	0.01	0.01
query8	0.03	0.03	0.03
query9	0.55	0.51	0.52
query10	0.56	0.56	0.57
query11	0.13	0.09	0.10
query12	0.13	0.10	0.10
query13	0.58	0.57	0.57
query14	0.73	0.77	0.75
query15	0.84	0.79	0.81
query16	0.37	0.37	0.37
query17	0.98	0.96	0.98
query18	0.25	0.25	0.27
query19	1.78	1.74	1.69
query20	0.02	0.01	0.01
query21	15.43	0.66	0.65
query22	3.05	4.30	2.41
query23	17.39	1.01	1.02
query24	2.14	0.37	0.19
query25	0.23	0.06	0.05
query26	0.18	0.13	0.13
query27	0.03	0.03	0.04
query28	12.60	0.84	0.83
query29	12.55	3.30	3.40
query30	0.61	0.58	0.56
query31	2.80	0.34	0.33
query32	3.39	0.44	0.44
query33	2.76	2.78	2.81
query34	15.48	4.29	4.29
query35	4.29	4.30	4.29
query36	1.09	1.02	1.01
query37	0.07	0.05	0.05
query38	0.05	0.03	0.03
query39	0.02	0.01	0.02
query40	0.17	0.13	0.13
query41	0.07	0.02	0.02
query42	0.02	0.01	0.02
query43	0.03	0.03	0.03
Total cold run time: 105.15 s
Total hot run time: 30.63 s

@sollhui
Copy link
Contributor Author

sollhui commented Mar 11, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 34.99% (8556/24452)
Line Coverage: 26.83% (69540/259230)
Region Coverage: 26.07% (36078/138392)
Branch Coverage: 23.04% (18446/80060)
Coverage Report: http://coverage.selectdb-in.cc/coverage/98fa69537fbd7b113e74913298b892e2c39388f1_98fa69537fbd7b113e74913298b892e2c39388f1/report/index.html

@doris-robot
Copy link

TPC-H: Total hot run time: 38135 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit 98fa69537fbd7b113e74913298b892e2c39388f1, data reload: false

------ Round 1 ----------------------------------
q1	17621	4109	4046	4046
q2	2043	158	153	153
q3	10547	940	929	929
q4	4669	939	940	939
q5	7609	2881	3013	2881
q6	178	129	124	124
q7	1283	829	816	816
q8	9476	2069	2064	2064
q9	7260	6515	6493	6493
q10	8210	2565	2555	2555
q11	414	235	216	216
q12	792	313	311	311
q13	17984	3006	2981	2981
q14	275	241	243	241
q15	499	461	455	455
q16	478	386	400	386
q17	946	905	891	891
q18	6741	5879	5885	5879
q19	1611	1521	1530	1521
q20	565	275	269	269
q21	7542	3710	3700	3700
q22	800	285	302	285
Total cold run time: 107543 ms
Total hot run time: 38135 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4010	4023	3989	3989
q2	330	215	225	215
q3	2957	2918	2945	2918
q4	1810	1815	1792	1792
q5	5236	5277	5220	5220
q6	202	117	116	116
q7	2257	1846	1844	1844
q8	3202	3283	3245	3245
q9	8541	8525	8542	8525
q10	6143	3720	3726	3720
q11	535	435	427	427
q12	709	558	540	540
q13	12145	2820	2828	2820
q14	267	241	245	241
q15	481	459	448	448
q16	442	409	403	403
q17	1693	1672	1669	1669
q18	7754	7546	7194	7194
q19	4100	1625	1607	1607
q20	1921	1668	1708	1668
q21	4881	4837	4765	4765
q22	525	430	429	429
Total cold run time: 70141 ms
Total hot run time: 53795 ms

@doris-robot
Copy link

TPC-DS: Total hot run time: 185873 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit 98fa69537fbd7b113e74913298b892e2c39388f1, data reload: false

query1	909	364	344	344
query2	7454	1935	1963	1935
query3	6711	220	210	210
query4	24997	20894	20759	20759
query5	4325	4462	460	460
query6	275	184	174	174
query7	4611	308	296	296
query8	232	169	163	163
query9	8427	2262	2258	2258
query10	452	239	243	239
query11	14904	14526	14241	14241
query12	128	87	84	84
query13	1654	455	464	455
query14	10629	10014	10024	10014
query15	250	193	190	190
query16	7788	264	266	264
query17	1904	570	543	543
query18	2040	276	266	266
query19	197	154	160	154
query20	94	85	85	85
query21	196	127	128	127
query22	4643	4448	4379	4379
query23	31343	30529	30729	30529
query24	11151	3080	3046	3046
query25	674	411	409	409
query26	1282	170	176	170
query27	2906	377	363	363
query28	6584	1843	1837	1837
query29	1023	604	609	604
query30	310	147	151	147
query31	949	712	737	712
query32	107	65	63	63
query33	756	254	257	254
query34	1064	465	478	465
query35	838	629	621	621
query36	930	840	831	831
query37	145	76	79	76
query38	3266	3166	3169	3166
query39	1403	1355	1395	1355
query40	207	111	113	111
query41	100	67	65	65
query42	106	97	105	97
query43	452	408	416	408
query44	1102	704	705	704
query45	280	253	249	249
query46	1056	809	777	777
query47	1618	1563	1552	1552
query48	411	348	350	348
query49	1147	328	348	328
query50	786	375	391	375
query51	6762	6781	6831	6781
query52	108	94	89	89
query53	345	279	283	279
query54	298	250	242	242
query55	84	83	83	83
query56	239	235	225	225
query57	1063	985	997	985
query58	242	221	210	210
query59	2685	2381	2681	2381
query60	280	256	278	256
query61	127	111	112	111
query62	647	417	416	416
query63	306	277	275	275
query64	5307	3410	3179	3179
query65	3225	3005	3004	3004
query66	1416	337	327	327
query67	14933	14367	14262	14262
query68	8426	563	590	563
query69	626	402	391	391
query70	1222	1072	1124	1072
query71	539	279	276	276
query72	6433	2647	2473	2473
query73	777	333	332	332
query74	6877	6535	6580	6535
query75	9709	8079	7966	7966
query76	5147	1106	1184	1106
query77	657	250	259	250
query78	10253	9568	9565	9565
query79	9575	511	516	511
query80	1473	402	387	387
query81	510	226	200	200
query82	579	196	194	194
query83	205	140	138	138
query84	289	80	80	80
query85	1362	341	322	322
query86	369	278	274	274
query87	3472	3257	3266	3257
query88	3607	2348	2343	2343
query89	545	354	407	354
query90	2055	181	184	181
query91	157	125	130	125
query92	64	48	50	48
query93	6650	539	520	520
query94	1242	184	182	182
query95	437	340	343	340
query96	595	261	259	259
query97	4396	4254	4264	4254
query98	231	229	210	210
query99	1180	780	758	758
Total cold run time: 302542 ms
Total hot run time: 185873 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 30.48 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit 98fa69537fbd7b113e74913298b892e2c39388f1, data reload: false

query1	0.04	0.03	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.68	0.10	0.10
query5	0.51	0.51	0.53
query6	1.33	0.67	0.68
query7	0.02	0.02	0.01
query8	0.04	0.03	0.02
query9	0.56	0.53	0.50
query10	0.57	0.56	0.56
query11	0.13	0.09	0.09
query12	0.13	0.11	0.10
query13	0.58	0.56	0.56
query14	0.75	0.76	0.75
query15	0.82	0.82	0.81
query16	0.37	0.37	0.37
query17	0.96	1.00	0.98
query18	0.27	0.26	0.26
query19	1.85	1.67	1.72
query20	0.01	0.02	0.01
query21	15.41	0.63	0.61
query22	2.89	4.40	2.23
query23	17.42	1.13	0.94
query24	2.14	0.21	0.36
query25	0.28	0.11	0.04
query26	0.18	0.14	0.14
query27	0.03	0.04	0.03
query28	12.51	0.83	0.82
query29	12.60	3.40	3.34
query30	0.57	0.57	0.55
query31	2.79	0.36	0.34
query32	3.35	0.43	0.44
query33	2.77	2.86	2.87
query34	15.52	4.29	4.30
query35	4.28	4.29	4.28
query36	1.10	1.02	1.02
query37	0.07	0.05	0.04
query38	0.04	0.03	0.03
query39	0.03	0.02	0.02
query40	0.17	0.14	0.13
query41	0.07	0.02	0.01
query42	0.03	0.02	0.02
query43	0.02	0.03	0.02
Total cold run time: 105.18 s
Total hot run time: 30.48 s

@sollhui
Copy link
Contributor Author

sollhui commented Mar 11, 2024

run buildall

@github-actions GitHub Actions
Copy link
Contributor

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TPC-H: Total hot run time: 37662 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpch-tools
Tpch sf100 test result on commit ff9232437c938ebac70c6e3b35e51e45467e87a9, data reload: false

------ Round 1 ----------------------------------
q1	17706	4045	3992	3992
q2	2029	146	140	140
q3	10638	945	926	926
q4	4675	925	913	913
q5	7597	2870	2967	2870
q6	192	124	125	124
q7	1278	812	798	798
q8	9510	1963	2066	1963
q9	7208	6477	6451	6451
q10	8218	2578	2538	2538
q11	416	213	215	213
q12	747	308	305	305
q13	17961	2977	2960	2960
q14	270	256	252	252
q15	494	471	458	458
q16	482	389	388	388
q17	937	860	852	852
q18	6638	5811	5825	5811
q19	1565	1508	1492	1492
q20	565	274	272	272
q21	7519	3662	3652	3652
q22	804	292	300	292
Total cold run time: 107449 ms
Total hot run time: 37662 ms

----- Round 2, with runtime_filter_mode=off -----
q1	4032	3973	3990	3973
q2	325	227	226	226
q3	2963	2896	2901	2896
q4	1823	1822	1805	1805
q5	5184	5234	5191	5191
q6	203	114	118	114
q7	2241	1836	1821	1821
q8	3195	3262	3246	3246
q9	8524	8489	8522	8489
q10	6119	3754	3712	3712
q11	534	434	435	434
q12	688	513	581	513
q13	10545	2844	2846	2844
q14	268	244	271	244
q15	483	446	447	446
q16	444	409	412	409
q17	1688	1655	1662	1655
q18	7723	7449	7223	7223
q19	5359	1606	1603	1603
q20	1851	1692	1680	1680
q21	4911	4710	4790	4710
q22	486	444	446	444
Total cold run time: 69589 ms
Total hot run time: 53678 ms

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 34.99% (8556/24452)
Line Coverage: 26.82% (69526/259236)
Region Coverage: 26.07% (36073/138395)
Branch Coverage: 23.04% (18443/80064)
Coverage Report: http://coverage.selectdb-in.cc/coverage/ff9232437c938ebac70c6e3b35e51e45467e87a9_ff9232437c938ebac70c6e3b35e51e45467e87a9/report/index.html

@doris-robot
Copy link

TPC-DS: Total hot run time: 185688 ms 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/tpcds-tools
TPC-DS sf100 test result on commit ff9232437c938ebac70c6e3b35e51e45467e87a9, data reload: false

query1	914	354	340	340
query2	7453	1941	1821	1821
query3	6704	215	217	215
query4	24747	20794	20816	20794
query5	4307	1151	486	486
query6	266	180	178	178
query7	4618	311	303	303
query8	244	169	176	169
query9	8376	2273	2248	2248
query10	449	241	258	241
query11	15100	14328	14395	14328
query12	146	86	86	86
query13	1650	417	443	417
query14	10387	9669	9730	9669
query15	239	205	187	187
query16	7759	277	276	276
query17	1896	584	560	560
query18	1975	294	277	277
query19	202	161	159	159
query20	98	90	89	89
query21	201	129	126	126
query22	4552	4436	4392	4392
query23	31134	30598	30780	30598
query24	11128	3052	3002	3002
query25	660	387	378	378
query26	1303	163	164	163
query27	2949	375	359	359
query28	6577	1860	1815	1815
query29	1040	607	614	607
query30	304	149	146	146
query31	930	744	735	735
query32	96	59	60	59
query33	759	271	264	264
query34	1010	463	472	463
query35	850	657	652	652
query36	948	803	841	803
query37	142	78	82	78
query38	3305	3127	3141	3127
query39	1438	1362	1385	1362
query40	213	120	137	120
query41	65	66	63	63
query42	104	102	96	96
query43	427	403	411	403
query44	1085	716	708	708
query45	277	276	261	261
query46	1049	788	768	768
query47	1629	1547	1558	1547
query48	415	353	346	346
query49	1139	330	338	330
query50	773	376	391	376
query51	6765	6695	6696	6695
query52	100	95	90	90
query53	351	286	289	286
query54	283	259	246	246
query55	84	85	86	85
query56	245	221	220	220
query57	1073	1021	1013	1013
query58	253	221	220	220
query59	2698	2423	2423	2423
query60	278	259	300	259
query61	112	110	113	110
query62	660	418	409	409
query63	307	284	286	284
query64	5384	3384	3476	3384
query65	3032	3020	3010	3010
query66	1422	323	338	323
query67	14814	14296	14007	14007
query68	9047	562	579	562
query69	612	384	374	374
query70	1205	1124	1089	1089
query71	519	284	276	276
query72	6445	2666	2530	2530
query73	868	322	323	322
query74	6888	6513	6454	6454
query75	9528	8011	7977	7977
query76	5497	1150	1175	1150
query77	678	263	246	246
query78	10201	9648	9666	9648
query79	8552	520	510	510
query80	1427	458	380	380
query81	494	212	216	212
query82	325	193	202	193
query83	222	151	139	139
query84	291	76	79	76
query85	1076	340	321	321
query86	357	295	284	284
query87	3419	3242	3244	3242
query88	3675	2368	2360	2360
query89	525	368	369	368
query90	2161	178	174	174
query91	165	128	128	128
query92	60	49	49	49
query93	6519	541	517	517
query94	1233	191	187	187
query95	453	342	361	342
query96	601	277	262	262
query97	4443	4283	4338	4283
query98	232	207	219	207
query99	1150	789	772	772
Total cold run time: 300761 ms
Total hot run time: 185688 ms

@doris-robot
Copy link

ClickBench: Total hot run time: 31.62 s 
machine: 'aliyun_ecs.c7a.8xlarge_32C64G'
scripts: https://github.com/apache/doris/tree/master/tools/clickbench-tools
ClickBench test result on commit ff9232437c938ebac70c6e3b35e51e45467e87a9, data reload: false

query1	0.03	0.03	0.03
query2	0.06	0.03	0.02
query3	0.23	0.06	0.06
query4	1.71	0.11	0.11
query5	0.52	0.51	0.50
query6	1.29	0.67	0.67
query7	0.01	0.01	0.01
query8	0.03	0.02	0.03
query9	0.57	0.52	0.52
query10	0.56	0.57	0.58
query11	0.12	0.10	0.10
query12	0.13	0.11	0.10
query13	0.58	0.57	0.57
query14	0.75	0.76	0.75
query15	0.83	0.82	0.81
query16	0.37	0.38	0.37
query17	0.99	0.99	1.02
query18	0.27	0.27	0.26
query19	1.84	1.81	1.81
query20	0.01	0.01	0.01
query21	17.66	0.66	0.68
query22	5.05	4.67	2.97
query23	17.85	1.02	1.01
query24	2.35	0.62	0.45
query25	0.34	0.04	0.04
query26	0.16	0.16	0.15
query27	0.04	0.03	0.03
query28	12.15	0.85	0.85
query29	12.54	3.21	3.17
query30	0.63	0.59	0.60
query31	2.79	0.34	0.33
query32	3.40	0.44	0.43
query33	2.85	2.79	2.82
query34	15.48	4.30	4.27
query35	4.29	4.34	4.30
query36	1.09	1.03	1.01
query37	0.07	0.05	0.05
query38	0.04	0.03	0.03
query39	0.02	0.02	0.02
query40	0.16	0.13	0.13
query41	0.08	0.02	0.02
query42	0.02	0.02	0.02
query43	0.02	0.02	0.03
Total cold run time: 109.98 s
Total hot run time: 31.62 s

@doris-robot
Copy link

Load test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G'

Load test result on commit ff9232437c938ebac70c6e3b35e51e45467e87a9 with default session variables
Stream load json:         18 seconds loaded 2358488459 Bytes, about 124 MB/s
Stream load orc:          59 seconds loaded 1101869774 Bytes, about 17 MB/s
Stream load parquet:      31 seconds loaded 861443392 Bytes, about 26 MB/s
Insert into select:       17.6 seconds inserted 10000000 Rows, about 568K ops/s

liaoxin01
liaoxin01 approved these changes Mar 12, 2024
View reviewed changes
Copy link
Contributor

@liaoxin01 liaoxin01 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Mar 12, 2024
@github-actions GitHub Actions
Copy link
Contributor

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

PR approved by anyone and no changes requested.

@yiguolei yiguolei merged commit 0bbcca4 into apache:master Mar 12, 2024
26 of 29 checks passed
yiguolei pushed a commit that referenced this pull request Mar 12, 2024
@sollhui @Doris-Extras
[opt](routine-load) end Kafka consume when meets partition EOF #32046
6610307
sollhui added a commit to sollhui/doris that referenced this pull request Jul 8, 2024
@sollhui
[opt](routine-load) end Kafka consume when meets partition EOF apache…
70947da 
…#32046
@sollhui sollhui mentioned this pull request Jul 8, 2024
Merged
dataroaring pushed a commit that referenced this pull request Jul 8, 2024
@sollhui
[opt](routine-load) end Kafka consume when meets partition EOF #32046 (…
c97f799 
…#37442)

pick #32046

Co-authored-by: HHoflittlefish777 <[email protected]>
@xiaokang xiaokang mentioned this pull request Jul 14, 2024
Closed
mongo360 pushed a commit to mongo360/doris that referenced this pull request Aug 16, 2024
@sollhui
[opt](routine-load) end Kafka consume when meets partition EOF apache…
e358a56 
…#32046 (apache#37442)

pick apache#32046

Co-authored-by: HHoflittlefish777 <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@yiguolei yiguolei yiguolei approved these changes

@liaoxin01 liaoxin01 liaoxin01 approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/2.0.13-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
6 participants
@sollhui @doris-robot @yiguolei @liaoxin01 @xiaokang @dataroaring