[NUTCH-3012] SegmentReader when dumping with option -recode: NPE on unparsed documents

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.19
• Fix Version/s: 1.20
• Component/s: segment
• Patch Info: Patch Available
• Assignee: Sebastian Nagel
• Reporter: Sebastian Nagel
• Created: 09/Oct/23 06:11
• Updated: 13/Mar/24 14:50
• Resolved: 21/Oct/23 14:21

Description

SegmentReader when called with the flag -recode fails with a NPE when trying to stringify the raw content of unparsed documents:

$> bin/nutch readseg  -dump crawl/segments/20231009065431 crawl/segreader/20231009065431 -recode
...
2023-10-09 07:55:18,451 INFO mapreduce.Job: Task Id : attempt_1696825862783_0005_r_000000_0, Status : FAILED
Error: java.lang.NullPointerException: charset
        at java.base/java.lang.String.<init>(String.java:504)
        at java.base/java.lang.String.<init>(String.java:561)
        at org.apache.nutch.protocol.Content.toString(Content.java:297)
        at org.apache.nutch.segment.SegmentReader$InputCompatReducer.reduce(SegmentReader.java:189)

Issue Links

• links to: GitHub Pull Request #787