Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: BucketCache
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      We keep Bucket entry map in BucketCache. Below is the math for heapSize for the key , value into this map.
      BlockCacheKey
      ---------------
      String hfileName - Ref - 4
      long offset - 8
      BlockType blockType - Ref - 4
      boolean isPrimaryReplicaBlock - 1
      Total = align(12 (Object) + 17 )= 32

      BucketEntry
      ------------
      int offsetBase - 4
      int length - 4
      byte offset1 - 1
      byte deserialiserIndex - 1
      long accessCounter - 8
      BlockPriority priority - Ref - 4
      volatile boolean markedForEvict - 1
      AtomicInteger refCount - 16 + 4
      long cachedTime - 8
      Total = align(12 (Object) + 51) = 64

      ConcurrentHashMap Map.Entry - 40
      blocksByHFile ConcurrentSkipListSet Entry - 40

      Total = 32 + 64 + 80 = 176

      For 10 million blocks we will end up having 1.6GB of heap size.
      This jira aims to reduce this as much as possible

        Attachments

        1. HBASE-17819_new_V2.patch
          12 kB
          Anoop Sam John
        2. HBASE-17819_new_V1.patch
          12 kB
          Anoop Sam John
        3. HBASE-17819_new_V1.patch
          11 kB
          Anoop Sam John
        4. HBASE-17819_V3.patch
          23 kB
          Anoop Sam John
        5. HBASE-17819_V2.patch
          21 kB
          Anoop Sam John
        6. HBASE-17819_V1.patch
          20 kB
          Anoop Sam John

          Issue Links

            Activity

              People

              • Assignee:
                anoop.hbase Anoop Sam John
                Reporter:
                anoop.hbase Anoop Sam John
              • Votes:
                0 Vote for this issue
                Watchers:
                10 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: