[SPARK-29450] [SS] In streaming aggregation, metric for output rows is not measured in append mode

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.4.4, 3.0.0
• Fix Version/s: 2.4.5, 3.0.0
• Component/s: Structured Streaming
• Assignee: Jungtaek Lim
• Reporter: Jungtaek Lim
• Created: 13/Oct/19 07:11
• Updated: 12/Dec/22 18:10
• Resolved: 19/Dec/19 09:24

Description

The bug is reported in dev. mailing list. Credit to jlaskowski. Quoting here:
 

I've just noticed that the number of output rows metric of StateStoreSaveExec physical operator does not seem to be measured for Append output mode. In other words, whatever happens before or after StateStoreSaveExec operator the metric is always 0.
 
It is measured for the other modes - Complete and Update.

https://lists.apache.org/thread.html/eec318f7ff84c700bffc8bdb7c95c0dcaffb2494ac7ccd7a7c7c6588@%3Cdev.spark.apache.org%3E

Issue Links

• links to: GitHub Pull Request #26104
• links to: GitHub Pull Request #27209