Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-6784

Avoid rescan twice in HDFS CacheReplicationMonitor for one FS Op if it calls setNeedsRescan multiple times.

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 3.0.0-alpha1
    • Fix Version/s: None
    • Component/s: caching
    • Labels:
      None
    • Target Version/s:

      Description

      In HDFS CacheReplicationMonitor, rescan is expensive. Sometimes, setNeedsRescan is called multiple times, for example, in FSNamesystem#modifyCacheDirective, there are 3 times. In monitor thread of CacheReplicationMonitor, if it checks needsRescan is true, rescan will happen, but needsRescan is set to false before real scan. Meanwhile, the 2nd or 3rd time setNeedsResacn may set needsRescan to true. So after the scan finish, in next loop, a new rescan will be triggered, that's not necessary at all and inefficient for rescan twice.

        Attachments

          Activity

            People

            • Assignee:
              hitliuyi Yi Liu
              Reporter:
              hitliuyi Yi Liu
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: