[NUTCH-2544] Nutch 1.15 no longer compatible with AWS EMR and S3

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.15
• Fix Version/s: 1.15
• Component/s: fetcher, generator
• Assignee: Sebastian Nagel
• Reporter: Steven W
• Created: 23/Mar/18 19:39
• Updated: 01/Oct/19 14:29
• Resolved: 26/Apr/18 10:57

Description

Nutch 1.14 is working OK with AWS EMR and S3 storage, but NUTCH-2375 appears to have broken this.

Generator partitioning fails with Error: java.lang.NullPointerException at org.apache.nutch.crawl.URLPartitioner.getPartition(URLPartitioner.java:75)

Issue Links

• is caused by: NUTCH-2375 Upgrade the code base from org.apache.hadoop.mapred to org.apache.hadoop.mapreduce
• links to: GitHub Pull Request #320