[NUTCH-1959] Improving CommonCrawlFormat implementations

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 1.9
• Fix Version/s: 1.10
• Assignee: Chris A. Mattmann
• Reporter: Giuseppe Totaro
• Created: 11/Mar/15 17:04
• Updated: 13/Mar/24 14:51
• Resolved: 26/Mar/15 04:36

Description

CommonCrawlFormat is an interface for Java classes that implement methods for writing data into Common Crawl format. AbstractCommonCrawlFormat is an abstract class that implements CommonCrawlFormat and provides abstract methods for "CommonCrawl formatter" classes.
You can find in attachment a PATCH that includes some improvements for CommonCrawlFormat-based classes;

• CommonCrawlFormat and AbstractCommonCrawlFormat now provide only the getJsonData() method, responsible for getting out JSON data.
• AbstractCommonCrawlFormat provides also the abstract methods that each subclass has to implement in order to handle JSON objects.
• CommonCrawlFormatSimple is a StringBuilder-based formatter that now provide also escaping of JSON string values.

This PATCH aims at providing a better interface for implementing/extending CommonCrawlFormat classes.

I would really appreciate your feedback.
Thanks a lot,
Giuseppe

Issue Links

• is part of: NUTCH-1974 keyPrefix option for CommonCrawlDataDumper tool