[NUTCH-2389] Precise data parsing using Jsoup CSS selectors

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.3
• Fix Version/s: 2.4
• Component/s: parser
• Assignee: Kaidul Islam
• Reporter: Kaidul Islam
• Due: 26/May/17
• Created: 23/May/17 09:30
• Updated: 13/Mar/24 14:51
• Resolved: 30/Jul/17 16:39

Description

As far as I know, currently Nutch 1.x and 2.x has no features to extract/parse exact contents for specific websites. I've developed a plugin parse-jsoup using Jsoup for my current project to extract precise content for site specific crawling using detailed XML configuration(field name, CSS-selector, attribute, extraction rules, data-type, default-value etc).

Please let me know if this feature seems relevant and currently not present in Nutch. I have also plan to export it into Nutch 1.x.