Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-1199

SortedKeyValueFile$Writer.append method have a puzzle for the sorted key

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.7.3
    • 1.7.3
    • java
    • Reviewed

    Description

      At the SortedKeyValueFile.java 539 lines like next:
      mPreviousKey = key;

      This class is same as Hadoop's MapFile, at the MapFile the same methon will keep a copy for this key, but not use "=".

      If use "=" at here, when user append a reuse key object to this file. will cause key sorted not valid.

      I think next code will fix it:

      private DataInputBuffer inBuf = new DataInputBuffer();
      private DataOutputBuffer outBuf = new DataOutputBuffer();

      GenericDatumWriter<K> writer = new ReflectDatumWriter<K>(schema);
      GenericDatumReader<K> reader = new ReflectDatumReader<K>(schema);
      writer.write(key, encoder);
      inBuf.reset(outBuf.getData(), outBuf.getLength());
      BinaryDecoder decoder = DecoderFactory.get().directBinaryDecoder(inBuf,
      null);
      lastKey = reader.read(null, decoder);

      Attachments

        1. AVRO-1199.path
          1 kB
          Libing Sun
        2. AVRO-1199.patch
          8 kB
          Doug Cutting

        Activity

          People

            cutting Doug Cutting
            kimslbing Libing Sun
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: