Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-14541

When evictableMmapped or evictable size is zero, do not throw NoSuchElementException

    XMLWordPrintableJSON

    Details

    • Hadoop Flags:
      Reviewed

      Description

      Our XiaoMi HBase team are evaluating the performence improvement of HBASE-21879, and we have few CPU flame graph & heap flame graph by using async-profiler, and find that there're some performence issues in DFSClient .

      See the attached two flame graphs, we can conclude that the try catch block in ShortCircuitCache#trimEvictionMaps has some serious perf problem , we should remove the try catch from DFSClient.

        /**
         * Trim the eviction lists.
         */
        private void trimEvictionMaps() {
          long now = Time.monotonicNow();
          demoteOldEvictableMmaped(now);
      
          while (true) {
            long evictableSize = evictable.size();
            long evictableMmappedSize = evictableMmapped.size();
            if (evictableSize + evictableMmappedSize <= maxTotalSize) {
              return;
            }
            ShortCircuitReplica replica;
            try {
              if (evictableSize == 0) {
                replica = (ShortCircuitReplica)evictableMmapped.get(evictableMmapped
                    .firstKey());
              } else {
                replica = (ShortCircuitReplica)evictable.get(evictable.firstKey());
              }
            } catch (NoSuchElementException e) {
              break;
            }
            if (LOG.isTraceEnabled()) {
              LOG.trace(this + ": trimEvictionMaps is purging " + replica +
                  StringUtils.getStackTrace(Thread.currentThread()));
            }
            purge(replica);
          }
        }
      

      Our Xiaomi HDFS Team member Lisheng Sun will prepare patch for this issue.

        Attachments

        1. HDFS-14541.002.patch
          3 kB
          Lisheng Sun
        2. HDFS-14541.001.patch
          3 kB
          Lisheng Sun
        3. HDFS-14541.000.patch
          3 kB
          Lisheng Sun
        4. before-QPS.png
          404 kB
          Zheng Hu
        5. before-heap-flame-graph.svg
          373 kB
          Zheng Hu
        6. before-cpu-flame-graph.svg
          369 kB
          Zheng Hu
        7. async-prof-pid-94152-cpu-1.svg
          366 kB
          Zheng Hu
        8. async-prof-pid-94152-alloc-2.svg
          384 kB
          Zheng Hu
        9. after-QPS.png
          272 kB
          Zheng Hu
        10. after-heap-flame-graph.svg
          359 kB
          Zheng Hu
        11. after-cpu-flame-graph.svg
          378 kB
          Zheng Hu

          Issue Links

            Activity

              People

              • Assignee:
                leosun08 Lisheng Sun
                Reporter:
                openinx Zheng Hu
              • Votes:
                1 Vote for this issue
                Watchers:
                18 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: