[HUDI-917] Calculation of 'stats wa' need to be modified

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: cli
• Assignee: Unassigned
• Reporter: yaojingyi
• Created: 21/May/20 06:10
• Updated: 18/May/22 10:33

Description

'Write Amplification Factor' = 'Total Written' / 'Total Upserted' 

'Total Written' is always increase.

'Total Upserted' is not always increase. When the newest commit is insert new data,'Total Upserted' will be 0.

It leads the result are not in line with our understanding. 

 

if I insert 3 times, update 1 times, then insert again(10 rows per time)

'stats wa ' result as follows:

 

'Total Written' need change to get the difference between adjacent commit.

I found that the numsWrites always increase. It's the reason of this.

 

I'll try to fix it.