Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[fix](regression) spare .testfile to make disk checker happy when injecting fault #29477

Merged
merged 1 commit into from
Jan 4, 2024
Merged

[fix](regression) spare .testfile to make disk checker happy when injecting fault #29477

merged 1 commit into from
Jan 4, 2024

Conversation

freemandealer
Copy link
Contributor

Proposed changes

Issue Number: close #xxx

Further comments

If this is a relatively large or complex change, kick off the discussion at [email protected] by explaining why you chose the solution you did and what alternatives you considered, etc...

@freemandealer
Copy link
Contributor Author

run buildall

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 3, 2024

clang-tidy review says "All clean, LGTM! 👍"

@doris-robot
Copy link

TeamCity be ut coverage result:
Function Coverage: 36.62% (8615/23524)
Line Coverage: 28.67% (70012/244165)
Region Coverage: 27.66% (36241/131038)
Branch Coverage: 24.35% (18511/76018)
Coverage Report: http://coverage.selectdb-in.cc/coverage/da224cee56207af15d19a45c7c343b60008ff94d_da224cee56207af15d19a45c7c343b60008ff94d/report/index.html

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit da224cee56207af15d19a45c7c343b60008ff94d, data reload: false

------ Round 1 ----------------------------------
q1	18740	5209	5138	5138
q2	2027	154	142	142
q3	10609	1070	1287	1070
q4	10192	808	822	808
q5	7841	2951	2900	2900
q6	212	139	137	137
q7	920	550	536	536
q8	9280	2070	2055	2055
q9	6898	6448	6432	6432
q10	8285	3086	3035	3035
q11	423	212	218	212
q12	403	234	238	234
q13	17999	3638	3616	3616
q14	248	217	218	217
q15	592	547	530	530
q16	452	415	392	392
q17	987	498	495	495
q18	7471	6748	6749	6748
q19	1617	1356	1263	1263
q20	734	351	351	351
q21	2817	2398	2470	2398
q22	374	342	321	321
Total cold run time: 109121 ms
Total hot run time: 39030 ms

----- Round 2, with runtime_filter_mode=off -----
q1	5109	5148	5119	5119
q2	351	251	251	251
q3	3325	3304	3292	3292
q4	2114	1988	2020	1988
q5	5835	5832	5814	5814
q6	216	129	128	128
q7	2377	1964	1902	1902
q8	3386	3443	3473	3443
q9	8858	8753	8806	8753
q10	3783	3884	3843	3843
q11	595	498	502	498
q12	801	629	645	629
q13	8653	3191	3175	3175
q14	293	270	287	270
q15	594	537	535	535
q16	566	520	511	511
q17	2008	1786	1771	1771
q18	8671	8452	8306	8306
q19	1635	1566	1605	1566
q20	2212	1985	1971	1971
q21	5636	5340	5290	5290
q22	594	496	509	496
Total cold run time: 67612 ms
Total hot run time: 59551 ms

@doris-robot
Copy link

TPC-DS test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpcds-tools

TPC-DS sf100 test result on commit da224cee56207af15d19a45c7c343b60008ff94d, data reload: false

run tpcds-sf100 query with default conf and session variables
query1	933	351	353	351
query2	6439	1898	1966	1898
query3	6652	206	202	202
query4	26436	22372	22421	22372
query5	4259	546	554	546
query6	276	182	184	182
query7	4580	292	278	278
query8	244	201	196	196
query9	8180	2680	2626	2626
query10	379	242	227	227
query11	16173	15480	15604	15480
query12	138	81	75	75
query13	1642	324	313	313
query14	12311	7166	7185	7166
query15	252	186	196	186
query16	6550	280	268	268
query17	1879	501	508	501
query18	1961	265	266	265
query19	287	143	136	136
query20	88	78	76	76
query21	191	101	95	95
query22	5161	4826	4595	4595
query23	32189	30994	31326	30994
query24	11906	2796	2800	2796
query25	581	356	350	350
query26	1725	154	143	143
query27	2856	279	273	273
query28	7009	1954	1935	1935
query29	2045	385	404	385
query30	297	143	145	143
query31	1016	786	780	780
query32	92	60	59	59
query33	739	268	262	262
query34	944	452	450	450
query35	889	773	771	771
query36	1380	1209	1218	1209
query37	192	69	73	69
query38	3386	3282	3302	3282
query39	1342	1305	1275	1275
query40	293	92	91	91
query41	38	37	36	36
query42	95	88	93	88
query43	528	520	525	520
query44	1064	714	726	714
query45	201	184	179	179
query46	1080	669	654	654
query47	1724	1565	1657	1565
query48	350	261	268	261
query49	1203	327	322	322
query50	728	367	333	333
query51	5394	5277	5342	5277
query52	91	93	84	84
query53	212	152	150	150
query54	1380	556	566	556
query55	97	91	86	86
query56	212	199	197	197
query57	994	968	961	961
query58	227	204	202	202
query59	2870	2682	2744	2682
query60	255	237	237	237
query61	94	88	88	88
query62	618	463	478	463
query63	172	154	147	147
query64	5914	1715	1669	1669
query65	3326	3267	3269	3267
query66	1389	342	334	334
query67	15667	15522	15380	15380
query68	12572	529	544	529
query69	518	255	248	248
query70	1737	1546	1458	1458
query71	487	229	221	221
query72	5701	3676	3561	3561
query73	2897	314	315	314
query74	6979	6373	6367	6367
query75	4986	2274	2261	2261
query76	6287	1156	1147	1147
query77	655	249	266	249
query78	9205	8788	8618	8618
query79	1164	515	495	495
query80	693	381	372	372
query81	472	210	212	210
query82	207	101	103	101
query83	179	142	133	133
query84	248	54	52	52
query85	942	282	270	270
query86	387	374	381	374
query87	3581	3394	3364	3364
query88	3048	2258	2282	2258
query89	342	269	272	269
query90	1992	209	196	196
query91	121	88	94	88
query92	57	53	53	53
query93	1723	499	501	499
query94	911	193	188	188
query95	479	404	409	404
query96	638	317	314	314
query97	4291	4134	4171	4134
query98	212	186	192	186
query99	1108	851	887	851
Total cold run time: 294634 ms
Total hot run time: 179537 ms

@doris-robot
Copy link

TPC-H test result on machine: 'aliyun_ecs.c7a.8xlarge_32C64G', run with scripts in https://github.com/apache/doris/tree/master/tools/tpch-tools

Tpch sf100 test result on commit da224cee56207af15d19a45c7c343b60008ff94d, data reload: false

run tpch-sf100 query with default conf and session variables
q1	5440	5109	5086	5086
q2	400	183	159	159
q3	1481	1174	1186	1174
q4	1081	849	769	769
q5	3182	3101	3123	3101
q6	224	146	135	135
q7	993	559	535	535
q8	2150	2286	2264	2264
q9	6726	6690	6677	6677
q10	3167	3103	3109	3103
q11	354	217	227	217
q12	387	239	252	239
q13	4431	3644	3692	3644
q14	251	210	207	207
q15	610	556	551	551
q16	458	425	408	408
q17	1047	541	586	541
q18	7088	6748	6792	6748
q19	1640	1529	1472	1472
q20	546	383	337	337
q21	2882	2482	2534	2482
q22	411	315	318	315
Total cold run time: 44949 ms
Total hot run time: 40164 ms

run tpch-sf100 query with default conf and set session variable runtime_filter_mode=off
q1	5090	5050	5027	5027
q2	335	254	243	243
q3	3388	3325	3321	3321
q4	2136	2043	2023	2023
q5	5961	5914	5955	5914
q6	222	125	131	125
q7	2368	1918	1899	1899
q8	3551	3641	3692	3641
q9	9092	9028	9030	9028
q10	3876	3907	3946	3907
q11	588	519	496	496
q12	790	681	642	642
q13	3881	3215	3199	3199
q14	292	281	272	272
q15	609	544	546	544
q16	560	492	530	492
q17	2012	1810	1798	1798
q18	8760	8371	8341	8341
q19	1739	1677	1648	1648
q20	2305	1979	1984	1979
q21	5669	5207	5302	5207
q22	575	481	495	481
Total cold run time: 63799 ms
Total hot run time: 60227 ms

@doris-robot
Copy link

(From new machine)TeamCity pipeline, clickbench performance test result:
the sum of best hot time: 46.93 seconds
stream load tsv: 565 seconds loaded 74807831229 Bytes, about 126 MB/s
stream load json: 19 seconds loaded 2358488459 Bytes, about 118 MB/s
stream load orc: 66 seconds loaded 1101869774 Bytes, about 15 MB/s
stream load parquet: 32 seconds loaded 861443392 Bytes, about 25 MB/s
insert into select: 28.0 seconds inserted 10000000 Rows, about 357K ops/s
storage size: 17183454861 Bytes

@freemandealer
Copy link
Contributor Author

run pipelinex_p0

dataroaring
dataroaring approved these changes Jan 3, 2024
View reviewed changes
Copy link
Contributor

@dataroaring dataroaring left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@github-actions github-actions bot added the approved Indicates a PR has been approved by one committer. label Jan 3, 2024
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 3, 2024

PR approved by at least one committer and no changes requested.

@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jan 3, 2024

PR approved by anyone and no changes requested.

@freemandealer
Copy link
Contributor Author

run pipelinex_p0

@dataroaring dataroaring merged commit 329d20d into apache:master Jan 4, 2024
28 of 30 checks passed
jackwener pushed a commit to jackwener/doris that referenced this pull request Jan 4, 2024
@freemandealer @jackwener
[fix](regression) spare .testfile to make disk checker happy when inj…
0188c29 
…ecting fault (apache#29477)

Signed-off-by: freemandealer <[email protected]>
HappenLee pushed a commit to HappenLee/incubator-doris that referenced this pull request Jan 12, 2024
@freemandealer @HappenLee
[fix](regression) spare .testfile to make disk checker happy when inj…
17ca9b5 
…ecting fault (apache#29477)

Signed-off-by: freemandealer <[email protected]>
@liaoxin01 liaoxin01 mentioned this pull request Jan 16, 2024
Merged
wsjz pushed a commit to wsjz/incubator-doris that referenced this pull request Feb 4, 2024
@freemandealer @wsjz
[fix](regression) spare .testfile to make disk checker happy when inj…
705ee7f 
…ecting fault (apache#29477)

Signed-off-by: freemandealer <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@Yukang-Lian Yukang-Lian Yukang-Lian approved these changes

@dataroaring dataroaring dataroaring approved these changes

Assignees
No one assigned
Labels
approved Indicates a PR has been approved by one committer. dev/3.0.0-merged reviewed
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@freemandealer @doris-robot @Yukang-Lian @dataroaring