Details
-
Improvement
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Reviewed
Description
For each pass of the sort and merge, I would like a count of the number of records. So for example, if the map output 100 records and they were sorted once, the counter would be 100. If it spilled twice and was merged together, it would be 200. Clearly in a multi-level merge, it may not be a multiple of the number of map output records. This would let the users easily see if they have values like io.sort.mb or io.sort.factor set too low.