Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-27257

Remove unnecessary usage of CachedBlocksByFile from RS UI

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.5.0, 3.0.0-alpha-4, 2.4.14
    • None
    • None

    Description

      We have started deploying regionservers with large BucketCaches, with over 1M blocks in the cache. I noticed that our RS UI was loading pretty slowly, and I also noticed the following warning:

      The stats below are incomplete! We ran into our accounting limit of 1000000 blocks. Up the configuration hbase.ui.blockcache.by.file.max.

      Tracking that down, it seems like the original implementation back in 2014 for the BlockCache statistics used CachedBlocksByFile to calculate counts, sizes, etc. But over time this was improved to be backed by CacheStats, for example in HBASE-11573, HBASE-15635, HBASE-18532, etc.

      At this point the cbsbf variable holding the CachedBlocksByFile object is not actually used by anything on the RS UI. Instead those per-file statistics have been moved to other endpoints, available via the "View block cache as JSON" and "as JSON by file" links.

      Removing the creation of the cbsbf variable makes a dramatic improvement on RS UI load times for large caches. Prior to the change, it consistently took 4-5s to load the UI. Afterward it takes a fraction of a second.

      Attachments

        1. async-prof-pid-29233-cpu-1.html
          186 kB
          Bryan Beaudreault

        Issue Links

          Activity

            People

              bbeaudreault Bryan Beaudreault
              bbeaudreault Bryan Beaudreault
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: