[FLINK-25081] When chaining an operator of a side output stream, the num records sent displayed on the dashboard is incorrect

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 1.14.0
• Component/s: Runtime / Metrics, Runtime / Task
• Assignee: Unassigned
• Reporter: Lijie Wang
• Created: 26/Nov/21 12:34
• Updated: 07/Dec/21 15:04
• Resolved: 07/Dec/21 15:04

Description

As show in the following figure, "Map" is an operator of a side output stream, the num records sent of first vertex is 0.

 

The job code is as follows:

final StreamExecutionEnvironment env = StreamExecutionEnvironment.getExecutionEnvironment();

SingleOutputStreamOperator<Long> dataStream =
        env.addSource(new DataGeneratorSource<>(RandomGenerator.longGenerator(1, 1000)))
                .returns(Long.class)
                .setParallelism(10)
                .slotSharingGroup("group1");

DataStream<Long> sideOutput = dataStream.getSideOutput(new OutputTag<Long>("10") {});
sideOutput.map(num -> num).setParallelism(10).slotSharingGroup("group1");

dataStream.addSink(new DiscardingSink<>()).setParallelism(10).slotSharingGroup("group2");

env.execute("WordCount"); 

Issue Links

• duplicates: FLINK-18808 Task-level numRecordsOut metric may be underestimated