Nutch
  1. Nutch
  2. NUTCH-1477

NPE when injecting with DataFileAvroStore

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Critical Critical
    • Resolution: Unresolved
    • Affects Version/s: 2.1
    • Fix Version/s: 2.5
    • Component/s: storage
    • Labels:
      None
    • Environment:

      Java 1.6.0_35

      Description

      Fresh installation of Nutch 2.1, configured to use DataFileAvroStore. Injection job throws NullPointerException, see below. No error when I switch to MemStore.

      java.lang.NullPointerException
      at org.apache.avro.io.BinaryEncoder.writeString(BinaryEncoder.java:133)
      at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:176)
      at org.apache.avro.generic.GenericDatumWriter.writeString(GenericDatumWriter.java:171)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:72)
      at org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:89)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:62)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
      at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:245)
      at org.apache.gora.avro.store.DataFileAvroStore.put(DataFileAvroStore.java:54)
      at org.apache.gora.mapreduce.GoraRecordWriter.write(GoraRecordWriter.java:60)
      at org.apache.hadoop.mapred.MapTask$NewDirectOutputCollector.write(MapTask.java:639)
      at org.apache.hadoop.mapreduce.TaskInputOutputContext.write(TaskInputOutputContext.java:80)
      at org.apache.nutch.crawl.InjectorJob$UrlMapper.map(InjectorJob.java:185)
      at org.apache.nutch.crawl.InjectorJob$UrlMapper.map(InjectorJob.java:85)
      at org.apache.hadoop.mapreduce.Mapper.run(Mapper.java:144)
      at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:764)
      at org.apache.hadoop.mapred.MapTask.run(MapTask.java:370)
      at org.apache.hadoop.mapred.LocalJobRunner$Job.run(LocalJobRunner.java:212)

      1. webpage.avsc
        2 kB
        Julien Nioche
      2. webpage.avsc
        2 kB
        Lewis John McGibbney
      3. webpage.avsc
        2 kB
        Alfonso Nishikawa
      4. webpage.avsc
        2 kB
        Alfonso Nishikawa
      5. gora-core-0.2.1.jar
        147 kB
        Lewis John McGibbney
      6. NUTCH-1477.patch
        6 kB
        Lewis John McGibbney

        Issue Links

          Activity

          Lewis John McGibbney made changes -
          Fix Version/s 2.5 [ 12335562 ]
          Fix Version/s 2.4 [ 12324540 ]
          Julien Nioche made changes -
          Assignee Julien Nioche [ jnioche ]
          Henry Saputra made changes -
          Link This issue requires GORA-174 [ GORA-174 ]
          Lewis John McGibbney made changes -
          Fix Version/s 2.4 [ 12324540 ]
          Fix Version/s 2.3 [ 12324325 ]
          Gavin made changes -
          Link This issue depends upon GORA-183 [ GORA-183 ]
          Gavin made changes -
          Link This issue depends on GORA-183 [ GORA-183 ]
          Gavin made changes -
          Link This issue depends upon GORA-174 [ GORA-174 ]
          Gavin made changes -
          Link This issue depends on GORA-174 [ GORA-174 ]
          Lewis John McGibbney made changes -
          Fix Version/s 2.3 [ 12324325 ]
          Fix Version/s 2.2 [ 12323285 ]
          Lewis John McGibbney made changes -
          Attachment gora-core-0.2.1.jar [ 12559897 ]
          Attachment NUTCH-1477.patch [ 12559898 ]
          Alfonso Nishikawa made changes -
          Attachment webpage.avsc [ 12559893 ]
          Alfonso Nishikawa made changes -
          Attachment webpage.avsc [ 12559852 ]
          Lewis John McGibbney made changes -
          Attachment webpage.avsc [ 12559838 ]
          Alfonso Nishikawa made changes -
          Link This issue depends on GORA-183 [ GORA-183 ]
          Julien Nioche made changes -
          Link This issue depends on GORA-174 [ GORA-174 ]
          Julien Nioche made changes -
          Link This issue is related to NUTCH-842 [ NUTCH-842 ]
          Julien Nioche made changes -
          Priority Major [ 3 ] Critical [ 2 ]
          Julien Nioche made changes -
          Attachment webpage.avsc [ 12550773 ]
          Julien Nioche made changes -
          Field Original Value New Value
          Assignee Julien Nioche [ jnioche ]
          Fix Version/s 2.2 [ 12323285 ]
          Mike Baranczak created issue -

            People

            • Assignee:
              Unassigned
              Reporter:
              Mike Baranczak
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:

                Development