[NUTCH-2441] ARG_SEGMENT usage

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.13
• Fix Version/s: 1.15
• Component/s: metadata
• Patch Info: Patch Available
• Flags: Patch
• Assignee: Unassigned
• Reporter: Semyon Semyonov
• Created: 16/Oct/17 14:31
• Updated: 01/Oct/19 14:28
• Resolved: 18/Jan/18 17:51

Description

The class metadata/Nutch.java public static final String ARG_SEGMENT = "segment" is not used correctly. In some cases Fetcher and ParseSegment it is interpreted as a single segmenet, in others CrawlDb, LinkDb, IndexingJob as an array of segments. Such misunderstanding leads to inconsistency of usage of the parameter.

After a discussion with wastl-nagel the proposed solution is to allow the usage of both array and a string in all cases. That gives an opportunity to not introduce the broken changes.

A path is proposed.

*The question left is refactoring, all these five components share the same code(two versions of the same code to be precise). Shouldn't we extract a method and reduce duplicates? *

Issue Links

• links to: GitHub Pull Request #250