[FLINK-27133] Performance regression in serializerHeavyString

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Cannot Reproduce
• Affects Version/s: 1.15.0, 1.16.0
• Fix Version/s: 1.15.0, 1.16.0
• Component/s: API / Type Serialization System, Benchmarks
• Assignee: Matthias Pohl
• Reporter: Piotr Nowojski
• Created: 08/Apr/22 09:04
• Updated: 11/Apr/22 12:00
• Resolved: 11/Apr/22 11:53

Description

http://codespeed.dak8s.net:8000/timeline/#/?exe=1&ben=serializerHeavyString&extr=on&quarts=on&equid=off&env=2&revs=200

Suspected range: 5f21d15a09..caa296b813

caa296b813b [8 days ago] [FLINK-26957][runtime] Adds invariant to LocalDataOutputStream to verify that no operation is allowed on a closed stream [Matthias Pohl]
c50b0706237 [8 days ago] [FLINK-26957][runtime] Removes flush in FileSystemJobResultStore [Matthias Pohl]
098dc7eca31 [7 days ago] [hotfix] Fixes error in JavaDoc [Matthias Pohl]
116f297478f [7 days ago] [hotfix] Fixes typo in NonClosingOutputStreamDecorator [Matthias Pohl]
74ecbd13cf8 [7 days ago] [hotfix][runtime] Adds write method for consistency reasons [Matthias Pohl]
e031f82528b [5 days ago] [hotfix] Modify spelling error in IOUtils.java [lidefu]
a5a31de5b30 [4 days ago] [FLINK-26368] [kafka] Add setProperty method to KafkaSinkBuilder [Gyula Fora]
b9c874edae8 [4 days ago] [hotfix][metrics][docs] Update documentation [Chesnay Schepler]
3416b6440ac [6 days ago] [FLINK-27027][ci] Add default log file suffix [Chesnay Schepler]
0d105ac910d [6 days ago] [FLINK-27027][ci] Prevent creation of empty log files [Chesnay Schepler]
7ad47230f22 [4 days ago] [FLINK-27026][build] Upgrade checkstyle plugin [Chesnay Schepler]
db8522cfddb [4 days ago] [FLINK-27034][gcp][tests] Use testcontainers [Chesnay Schepler]
0097b5a6fae [3 weeks ago] [FLINK-26712][table-planner] Metadata keys should not conflict with physical columns [Timo Walther]
26aa543b3bb [3 months ago] [FLINK-25907][runtime][security] Add pluggable delegation token manager [gabor.g.somogyi]
cda343349f5 [7 days ago] [FLINK-26987][runtime] Fixes getAllAndLock livelock [Matthias Pohl]

I've run a benchmark request before FLINK-26957 and it suggests that it is indeed the cause for this regression:
http://codespeed.dak8s.net:8080/job/flink-benchmark-request/77/artifact/jmh-result.csv/*view*/

CC mapohl

Issue Links

• is caused by: FLINK-26957 FileSystemJobResultStore calls flush on an already closed OutputStream
• Testing discovered: FLINK-27165 Benchmark test SerializationFrameworkMiniBenchmarks#serializerHeavyString became unstable