Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-21879 Read HFile's block to ByteBuffer directly instead of to byte for reducing young gc purpose
  3. HBASE-22531

The HFileReaderImpl#shouldUseHeap return the incorrect true when disabled BlockCache

    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I'm having a benchmark with block cache disabled for HBASE-21879 branch. Just caurious about why still so many heap allocation in the heap allocation flame graph async-prof-pid-13311-alloc-4.svg , actually, I've set the following config, which means all allocation should be offheap, while it's not:

      # Disable the block cache
      hfile.block.cache.size=0
      hbase.ipc.server.reservoir.minimal.allocating.size=0   # Let all allocation from pooled allocator. 
      

      Checked the code, I found the problem here:

        private boolean shouldUseHeap(BlockType expectedBlockType) {
          if (cacheConf.getBlockCache() == null) {
            return false;
          } else if (!cacheConf.isCombinedBlockCache()) {
            // Block to cache in LruBlockCache must be an heap one. So just allocate block memory from
            // heap for saving an extra off-heap to heap copying.
            return true;
          }
          return expectedBlockType != null && !expectedBlockType.isData();
        }
      

      Say, the CacheConfig#getBlockCache will return a Optional<BlockCache>, which is always non-null:

        /**
         * Returns the block cache.
         *
         * @return the block cache, or null if caching is completely disabled
         */
        public Optional<BlockCache> getBlockCache() {
          return Optional.ofNullable(this.blockCache);
        }
      

        Attachments

        1. async-prof-pid-13311-alloc-4.svg
          196 kB
          Zheng Hu
        2. HBASE-22531.HBASE-21879.v1.patch
          4 kB
          Zheng Hu
        3. async-prof-pid-8590-alloc-2.svg
          354 kB
          Zheng Hu

          Issue Links

            Activity

              People

              • Assignee:
                openinx Zheng Hu
                Reporter:
                openinx Zheng Hu
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: