[YETUS-922] Precommit qualitative checks vote be -0 on overall improvement

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Not A Problem
• Component/s: Precommit
• Assignee: Unassigned
• Reporter: Nick Dimiduk
• Created: 30/Oct/19 18:42
• Updated: 05/Mar/20 16:23
• Resolved: 05/Mar/20 14:29

Description

Looking at the output over on hbase/775, I think the -1 votes on qualitative checks are a bit harsh. The tests I'm looking at are javac and {checkstyle}}, where we have a qualitative measure of change in quality. In this case, the patch improved quality by reducing the overall number of failure occurrences. I think these should be voted as -0 rather than -1. I suspect the reasoning behind the -1 vote is that the patch is viewed to have introduced new failures. The thing is, with patches that refactor code, this simple diff isn't able to distinguish between an actual new failure and a moved failure.

I could also argue that they should actually be +1 when total is less than previous because it's positive trajectory for the code base.

javac | hbase-server generated 1 new + 3 unchanged - 3 fixed = 4 total (was 6)
checkstyle | hbase-server: The patch generated 12 new + 270 unchanged - 37 fixed = 282 total (was 307)