[NUTCH-1585] Ensure duplicate tags do not exist in microformat-reltag tag set.

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.6, 2.2
• Fix Version/s: 1.7, 2.2.1
• Component/s: parser
• Assignee: Lewis John McGibbney
• Reporter: Lewis John McGibbney
• Created: 19/Jun/13 03:25
• Updated: 27/Jun/13 17:14
• Resolved: 27/Jun/13 17:14

Description

A WebPage can have many many embedded tags and other such markup.
Creating huge tag lists containing many many duplicates is counter productive to the process of parsing and extracting out such structure.
We should add a mechanism to only include single tag occurrences for the microformats-reltag parser.