[NUTCH-1495] -normalize and -filter for updatedb command in nutch 2.x

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Auto Closed
• Affects Version/s: 2.2
• Fix Version/s: 2.5
• Patch Info: Patch Available
• Assignee: Unassigned
• Reporter: Nathan Gass
• Created: 09/Nov/12 16:43
• Updated: 13/Oct/19 22:35
• Resolved: 13/Oct/19 22:35

Description

AFAIS in nutch 1.x you could change your url filters and normalizers during the crawl, and update the db using crawldb -normalize -filter. There does not seem to be a away to achieve the same in nutch 2.x?

Anyway, I went ahead and tried to implement -normalize and -filter for the nutch 2.x updatedb command. I have no experience with any of the used technologies including java, so please check the attached code carefully before using it. I'm very interested to hear if this is the right approach or any other comments.