Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7215 Put, Delete, Increment, Result, all all HBase M/R classes still implement/use Writable
  3. HBASE-7239

Verify protobuf serialization is correctly chunking upon read to avoid direct memory OOMs

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.95.1
    • Component/s: None
    • Labels:
      None

      Description

      Result.readFields() used to read from the input stream in 8k chunks to avoid OOM issues with direct memory.
      (Reading variable sized chunks into direct memory prevent the JVM from reusing the allocated direct memory and direct memory is only collected during full GCs)

      This is just to verify protobufs parseFrom type methods do the right thing as well so that we do not reintroduce this problem.

        Attachments

        1. 7239-1.patch
          2 kB
          Devaraj Das

          Issue Links

            Activity

              People

              • Assignee:
                ddas Devaraj Das
                Reporter:
                larsh Lars Hofhansl
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: