HBase
  1. HBase
  2. HBASE-3421

Very wide rows -- 30M plus -- cause us OOME

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.90.0
    • Fix Version/s: 0.90.5
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      A new config parameter, "hbase.hstore.compaction.kv.max", has been added to limit the number of rows processed in each iteration of the internal compaction code.
      Default value is 10.
      Show
      A new config parameter, "hbase.hstore.compaction.kv.max", has been added to limit the number of rows processed in each iteration of the internal compaction code. Default value is 10.

      Description

      From the list, see 'jvm oom' in http://mail-archives.apache.org/mod_mbox/hbase-user/201101.mbox/browser, it looks like wide rows – 30M or so – causes OOME during compaction. We should check it out. Can the scanner used during compactions use the 'limit' when nexting? If so, this should save our OOME'ing (or, we need to add to the next a max size rather than count of KVs).

      1. 3421.addendum
        0.7 kB
        Ted Yu
      2. HBASE-3421.patch
        0.8 kB
        Nate Putnam
      3. HBASE-34211-v2.patch
        1 kB
        Nate Putnam
      4. HBASE-34211-v3.patch
        2 kB
        Nate Putnam
      5. HBASE-34211-v4.patch
        1 kB
        Nate Putnam

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Nate Putnam
              Reporter:
              stack
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development