[NUTCH-2011] Endpoint to support realtime JSON output from the fetcher

• Type: Sub-task
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Component/s: fetcher, REST_api
• Labels: memex
• Assignee: Chris A. Mattmann
• Reporter: Sujen Shah
• Created: 15/May/15 13:04
• Updated: 02/Oct/15 08:57

Description

This fix will create an endpoint to query the Nutch REST service and get a real-time JSON response of the current/past Fetched URLs.

This endpoint also includes pagination of the output to reduce data transfer bw in large crawls.