[NUTCH-2574] Generator: hostCount >= maxCount comparison wrong

• Type: Bug
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.13
• Fix Version/s: 1.15
• Component/s: generator
• Assignee: Sebastian Nagel
• Reporter: Michael Coffey
• Created: 28/Apr/18 00:04
• Updated: 01/Oct/19 14:29
• Resolved: 13/Jun/18 08:39

Description

In the Generator.Selector.reduce function, there is a comparison of hostCount[1] to maxCount, to determine whether or not to push the current URL to the next segment. The purpose is to honor generate.max.count.

Sebastian noticed that it should test if (hostCount[1] > maxCount) rather than ">=".  As it stands, the code sometimes puts one less url into a segment than it should.

Issue Links

• links to: GitHub Pull Request #344