[NUTCH-2785] FreeGenerator: command-line option to define number of generated fetch lists

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.16
• Fix Version/s: 1.17
• Component/s: generator
• Patch Info: Patch Available
• Assignee: Sebastian Nagel
• Reporter: Sebastian Nagel
• Created: 28/Apr/20 17:50
• Updated: 28/Jan/21 13:15
• Resolved: 05/May/20 13:57

Description

While Generator allows to specify the number of generated fetch lists using the "-numFetchers" command-line option, FreeGenerator does not provide such an option. It uses the value of "mapreduce.job.maps" (2 by default), also in local mode where Generator always creates only one single fetch list.

Issue Links

• links to: GitHub Pull Request #519