[NUTCH-3059] Generator: selector job does not count reduce output records

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 1.20
• Fix Version/s: 1.21
• Component/s: generator
• Assignee: Unassigned
• Reporter: Sebastian Nagel
• Created: 05/Jun/24 12:17
• Updated: 14/Sep/24 14:48

Description

The selector step (job) of the Generator does not count the reduce output records resp. shows the count "0":

2024-06-05 13:57:09,299 INFO o.a.n.c.Generator [main] Generator: starting

2024-06-05 13:57:09,299 INFO o.a.n.c.Generator [main] Generator: selecting best-scoring urls due for fetch.
...
         Map-Reduce Framework
                Map input records=6
                Map output records=6
                ...
                Combine input records=0
                Combine output records=0
                Reduce input groups=1
                Reduce shuffle bytes=594
                Reduce input records=6
                Reduce output records=0
                Spilled Records=12
                ...

Not a big issue but should investigate why this happens. The other counters seem to work properly, also the partitioner job shows the reduce output records. The issue is observed in local and distributed mode.