[NUTCH-1477] NPE when injecting with DataFileAvroStore

• Type: Bug
• Status: Closed
• Priority: Critical
• Resolution: Auto Closed
• Affects Version/s: 2.1
• Fix Version/s: 2.5
• Component/s: storage

Environment

Java 1.6.0_35

• Assignee: Unassigned
• Reporter: Mike Baranczak
• Created: 11/Oct/12 01:11
• Updated: 03/Mar/22 14:27
• Resolved: 13/Oct/19 22:35

Description

Fresh installation of Nutch 2.1, configured to use DataFileAvroStore. Injection job throws NullPointerException, see below. No error when I switch to MemStore.

java.lang.NullPointerException
at org.apache.avro.io.BinaryEncoder.writeString(BinaryEncoder.java:133)
at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:176)
at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:171)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:89)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:245)
at org.apache.gora.avro.store.DataFileAvroStore.put(DataFileAvroStore.java:54)
at org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:60)
at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:639)
at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
at org.apache.nutch.crawl.InjectorJob$UrlMapper.map(InjectorJob.java:185)
at org.apache.nutch.crawl.InjectorJob$UrlMapper.map(InjectorJob.java:85)
at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

Issue Links

• depends upon: GORA-174 GORA compiler does not handle ["string", "null"] unions in the AVRO schema
• depends upon: GORA-183 dataStore.put() ->org.apache.gora.hbase.util.HBaseInterface#toBytes(). Unknown type: UNION
• is related to: NUTCH-842 AutoGenerate WebPage code
• requires: GORA-174 GORA compiler does not handle ["string", "null"] unions in the AVRO schema