HBase
  1. HBase
  2. HBASE-707

High-load import of data into single table/family never triggers split

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.1.3
    • Fix Version/s: 0.1.3
    • Component/s: None
    • Labels:
      None
    • Environment:

      Linux 2.6.25-14.fc9.x86_64, Fedora Core 9

      Description

      Importing a heavy amount of data into a single table and family.

      One column in that family (the same fam:col for every row) contains a frequently large amount of UTF-8 data. This column grows and grows but never causes a region split.

      Currently there is a single mapfile containing nearly 10GB.

      Eventually this has caused regions to crash with OOME, as described in HBASE-706

      Table in question:

      hql > describe items;
      -----------------------------------------------------------------------------

      Column Family Descriptor

      -----------------------------------------------------------------------------

      name: cfrecs, max versions: 2, compression: NONE, in memory: false, max leng
      th: 2147483647, bloom filter: none

      -----------------------------------------------------------------------------

      name: clusters, max versions: 2, compression: NONE, in memory: false, max le
      ngth: 2147483647, bloom filter: none

      -----------------------------------------------------------------------------

      name: content, max versions: 2, compression: NONE, in memory: false, max len
      gth: 2147483647, bloom filter: none

      -----------------------------------------------------------------------------

      name: readby, max versions: 2, compression: NONE, in memory: false, max leng
      th: 2147483647, bloom filter: none

      -----------------------------------------------------------------------------

      name: receivedby, max versions: 2, compression: NONE, in memory: false, max
      length: 2147483647, bloom filter: none

      -----------------------------------------------------------------------------

      name: savedby, max versions: 2, compression: NONE, in memory: false, max len
      gth: 2147483647, bloom filter: none

      -----------------------------------------------------------------------------

      name: sentby, max versions: 2, compression: NONE, in memory: false, max leng
      th: 2147483647, bloom filter: none

      -----------------------------------------------------------------------------
      7 columnfamily(s) in set. (0.34 sec)

        Issue Links

          Activity

          Jim Kellerman made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          stack made changes -
          Assignee stack [ stack ]
          stack made changes -
          Resolution Fixed [ 1 ]
          Status Open [ 1 ] Resolved [ 5 ]
          stack made changes -
          Attachment 707.patch [ 12384702 ]
          Jonathan Gray made changes -
          Description Importing a heavy amount of data into a single table and family.

          One column in that family (the same fam:col for every row) contains a frequently large amount of UTF-8 data. This column grows and grows but never causes a region split.

          Currently there is a single mapfile containing nearly 10GB.

          Eventually this has caused regions to crash with OOME, as described in HBASE-706
          Importing a heavy amount of data into a single table and family.

          One column in that family (the same fam:col for every row) contains a frequently large amount of UTF-8 data. This column grows and grows but never causes a region split.

          Currently there is a single mapfile containing nearly 10GB.

          Eventually this has caused regions to crash with OOME, as described in HBASE-706


          Table in question:

          hql > describe items;
          +-----------------------------------------------------------------------------+
          | Column Family Descriptor |
          +-----------------------------------------------------------------------------+
          | name: cfrecs, max versions: 2, compression: NONE, in memory: false, max leng|
          | th: 2147483647, bloom filter: none |
          +-----------------------------------------------------------------------------+
          | name: clusters, max versions: 2, compression: NONE, in memory: false, max le|
          | ngth: 2147483647, bloom filter: none |
          +-----------------------------------------------------------------------------+
          | name: content, max versions: 2, compression: NONE, in memory: false, max len|
          | gth: 2147483647, bloom filter: none |
          +-----------------------------------------------------------------------------+
          | name: readby, max versions: 2, compression: NONE, in memory: false, max leng|
          | th: 2147483647, bloom filter: none |
          +-----------------------------------------------------------------------------+
          | name: receivedby, max versions: 2, compression: NONE, in memory: false, max |
          | length: 2147483647, bloom filter: none |
          +-----------------------------------------------------------------------------+
          | name: savedby, max versions: 2, compression: NONE, in memory: false, max len|
          | gth: 2147483647, bloom filter: none |
          +-----------------------------------------------------------------------------+
          | name: sentby, max versions: 2, compression: NONE, in memory: false, max leng|
          | th: 2147483647, bloom filter: none |
          +-----------------------------------------------------------------------------+
          7 columnfamily(s) in set. (0.34 sec)
          Jonathan Gray made changes -
          Field Original Value New Value
          Link This issue relates to HBASE-706 [ HBASE-706 ]
          Jonathan Gray created issue -

            People

            • Assignee:
              stack
              Reporter:
              Jonathan Gray
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development