Hadoop HDFS
  1. Hadoop HDFS
  2. HDFS-6784

Avoid rescan twice in HDFS CacheReplicationMonitor for one FS Op if it calls setNeedsRescan multiple times.

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 3.0.0
    • Fix Version/s: None
    • Component/s: caching
    • Labels:
      None
    • Target Version/s:

      Description

      In HDFS CacheReplicationMonitor, rescan is expensive. Sometimes, setNeedsRescan is called multiple times, for example, in FSNamesystem#modifyCacheDirective, there are 3 times. In monitor thread of CacheReplicationMonitor, if it checks needsRescan is true, rescan will happen, but needsRescan is set to false before real scan. Meanwhile, the 2nd or 3rd time setNeedsResacn may set needsRescan to true. So after the scan finish, in next loop, a new rescan will be triggered, that's not necessary at all and inefficient for rescan twice.

        Activity

          People

          • Assignee:
            Yi Liu
            Reporter:
            Yi Liu
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development