Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-8391

Investigate AllocationFileLoaderService.reloadListener locking issue

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 3.2.0
    • Fix Version/s: 3.2.0
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Per findbugs report in YARN-8390, there is some inconsistent locking of  reloadListener

       

      Warnings

      Click on a warning row to see full context information.

      Multithreaded correctness Warnings

      Code Warning
      IS Inconsistent synchronization of org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadListener; locked 75% of time
        Bug type IS2_INCONSISTENT_SYNC (click for details)
      In class org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService
      Field org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AllocationFileLoaderService.reloadListener
      Synchronized 75% of the time
      Unsynchronized access at AllocationFileLoaderService.java:[line 117]
      Synchronized access at AllocationFileLoaderService.java:[line 212]
      Synchronized access at AllocationFileLoaderService.java:[line 228]
      Synchronized access at AllocationFileLoaderService.java:[line 269]

      Details

      IS2_INCONSISTENT_SYNC: Inconsistent synchronization

      The fields of this class appear to be accessed inconsistently with respect to synchronization.  This bug report indicates that the bug pattern detector judged that

      • The class contains a mix of locked and unlocked accesses,
      • The class is not annotated as javax.annotation.concurrent.NotThreadSafe,
      • At least one locked access was performed by one of the class's own methods, and
      • The number of unsynchronized field accesses (reads and writes) was no more than one third of all accesses, with writes being weighed twice as high as reads

      A typical bug matching this bug pattern is forgetting to synchronize one of the methods in a class that is intended to be thread-safe.

      You can select the nodes labeled "Unsynchronized access" to show the code locations where the detector believed that a field was accessed without synchronization.

      Note that there are various sources of inaccuracy in this detector; for example, the detector cannot statically detect all situations in which a lock is held.  Also, even when the detector is accurate in distinguishing locked vs. unlocked accesses, the code in question may still be correct.

        Attachments

        1. YARN-8191.001.patch
          2 kB
          Szilard Nemeth
        2. YARN-8391.002.patch
          1 kB
          Szilard Nemeth

          Issue Links

            Activity

              People

              • Assignee:
                snemeth Szilard Nemeth
                Reporter:
                haibochen Haibo Chen
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: