Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-7215 Put, Delete, Increment, Result, all all HBase M/R classes still implement/use Writable
  3. HBASE-7239

Verify protobuf serialization is correctly chunking upon read to avoid direct memory OOMs

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Closed
    • Critical
    • Resolution: Fixed
    • None
    • 0.95.1
    • None
    • None

    Description

      Result.readFields() used to read from the input stream in 8k chunks to avoid OOM issues with direct memory.
      (Reading variable sized chunks into direct memory prevent the JVM from reusing the allocated direct memory and direct memory is only collected during full GCs)

      This is just to verify protobufs parseFrom type methods do the right thing as well so that we do not reintroduce this problem.

      Attachments

        1. 7239-1.patch
          2 kB
          Devaraj Das

        Issue Links

          depends upon

          Task - A task that needs to be done. HBASE-7898 Serializing Cells over RPC

          • Blocker - Blocks development and/or testing work, production could not run
          • Closed

          Activity

            People

              ddas Devaraj Das
              larsh Lars Hofhansl
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: