[NUTCH-1504] Pluggable url partitioner

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.6
• Component/s: generator
• Patch Info: Patch Available
• Assignee: Lewis John McGibbney
• Reporter: Sourajit Basak
• Created: 17/Dec/12 17:38
• Updated: 28/Aug/15 20:34

Description

At present, the url partition logic is hard wired inside nutch core. It should be pluggable like FetchSchedule customized via nutch-site.xml.

There might be use cases where a single domain needs to be partioned on some custom logic. The existing UrlPartitioner cannot handle such cases.

Hence the requirement.