Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0, 0.94.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      This would add a new boolean config option: hfile.block.cache.datablocks
      Default would be true.

      Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the (aggregate) cache.
      This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets).

      I would like to get a general feeling about what folks think about this.
      The change itself would be simple.

      Update (Mikhail): we probably don't need a new conf option. Instead, we will make index blocks cached by default.

      1. 4683.txt
        9 kB
        Lars Hofhansl
      2. 4683-v2.txt
        10 kB
        Lars Hofhansl
      3. HBASE-4683-v3.patch
        3 kB
        Jean-Daniel Cryans
      4. ASF.LICENSE.NOT.GRANTED--D807.1.patch
        33 kB
        Phabricator
      5. ASF.LICENSE.NOT.GRANTED--D807.2.patch
        42 kB
        Phabricator
      6. HBASE-4683-0.92-v2.patch
        8 kB
        Jean-Daniel Cryans
      7. ASF.LICENSE.NOT.GRANTED--D807.3.patch
        42 kB
        Phabricator
      8. 0001-Cache-important-block-types.patch
        47 kB
        Mikhail Bautin
      9. ASF.LICENSE.NOT.GRANTED--D1695.1.patch
        8 kB
        Phabricator

        Issue Links

          Activity

          Lars Hofhansl created issue -
          Lars Hofhansl made changes -
          Field Original Value New Value
          Description This would add a new boolean config option: hfile.block.cache.datablocks
          Default would be true.

          Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the cache.
          This is the equivalent of setting all cacheBlocks to false on all scans (including scans on behalf of gets).

          I would like to general feeling about what folks think about this.
          The change itself would be simple.
          This would add a new boolean config option: hfile.block.cache.datablocks
          Default would be true.

          Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the cache.
          This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets).

          I would like to general feeling about what folks think about this.
          The change itself would be simple.
          Lars Hofhansl made changes -
          Description This would add a new boolean config option: hfile.block.cache.datablocks
          Default would be true.

          Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the cache.
          This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets).

          I would like to general feeling about what folks think about this.
          The change itself would be simple.
          This would add a new boolean config option: hfile.block.cache.datablocks
          Default would be true.

          Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the (aggregate) cache.
          This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets).

          I would like to get a general feeling about what folks think about this.
          The change itself would be simple.
          Lars Hofhansl made changes -
          Attachment 4683.txt [ 12501186 ]
          Lars Hofhansl made changes -
          Attachment 4683-v2.txt [ 12502694 ]
          Lars Hofhansl made changes -
          Assignee Lars Hofhansl [ lhofhansl ]
          Mikhail Bautin made changes -
          Link This issue relates to HBASE-5000 [ HBASE-5000 ]
          Mikhail Bautin made changes -
          Summary Create config option to only cache index blocks Always cache index blocks
          Mikhail Bautin made changes -
          Description This would add a new boolean config option: hfile.block.cache.datablocks
          Default would be true.

          Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the (aggregate) cache.
          This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets).

          I would like to get a general feeling about what folks think about this.
          The change itself would be simple.
          This would add a new boolean config option: hfile.block.cache.datablocks
          Default would be true.

          Setting this to false allows HBase in a mode where only index blocks are cached, which is useful for analytical scenarios where a useful working set of the data cannot be expected to fit into the (aggregate) cache.
          This is the equivalent of setting cacheBlocks to false on all scans (including scans on behalf of gets).

          I would like to get a general feeling about what folks think about this.
          The change itself would be simple.

          Update (Mikhail): we probably don't need a new conf option. Instead, we will make index blocks cached by default.
          Mikhail Bautin made changes -
          Assignee Lars Hofhansl [ lhofhansl ] Mikhail Bautin [ mikhail ]
          Jean-Daniel Cryans made changes -
          Attachment HBASE-4683-v3.patch [ 12507072 ]
          Mikhail Bautin made changes -
          Summary Always cache index blocks Always cache index and bloom blocks
          Phabricator made changes -
          Attachment D807.1.patch [ 12507117 ]
          Phabricator made changes -
          Attachment D807.2.patch [ 12507165 ]
          Jean-Daniel Cryans made changes -
          Fix Version/s 0.92.0 [ 12314223 ]
          Jean-Daniel Cryans made changes -
          Attachment HBASE-4683-0.92-v2.patch [ 12507270 ]
          Phabricator made changes -
          Attachment D807.3.patch [ 12507452 ]
          Mikhail Bautin made changes -
          Attachment 0001-Cache-important-block-types.patch [ 12507453 ]
          Jean-Daniel Cryans made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Resolution Fixed [ 1 ]
          Phabricator made changes -
          Attachment D1695.1.patch [ 12514074 ]
          Lars Hofhansl made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Mikhail Bautin
              Reporter:
              Lars Hofhansl
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development