[NUTCH-646] New Indexing Framework for Nutch

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.9.0
• Fix Version/s: 0.9.0, 1.0.0
• Component/s: indexer

Environment

All

• Patch Info: Patch Available
• Assignee: Dennis Kubes
• Reporter: Dennis Kubes
• Due: 18/Aug/08
• Created: 19/Aug/08 04:03
• Updated: 02/May/13 02:29
• Resolved: 04/Dec/08 21:29

Description

New indexing framework for Nutch that provides a more generic field abstraction consistent with Lucene index semantics. Allows multiple MR jobs to be created for different fields and those fields to be aggregated and indexed in the end. Overcomes limitations of the current indexer that limits what databases are passed into the indexer. Creates a new extension point as well for field-filters for manipulation of fields during the indexing process.

Issue Links

• depends upon: NUTCH-635 LinkAnalysis Tool for Nutch