Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2313

Livelock can occur in FairScheduler when there are lots of running apps

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.4.1
    • Fix Version/s: 2.6.0
    • Component/s: fairscheduler
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Observed livelock on FairScheduler when there are lots entry in queue. After my investigating code, following case can occur:

      1. update() called by UpdateThread takes longer times than UPDATE_INTERVAL(500ms) if there are lots queue.
      2. UpdateThread goes busy loop.
      3. Other threads(AllocationFileReloader, ResourceManager$SchedulerEventDispatcher) can wait forever.

        Attachments

        1. rm-stack-trace.txt
          138 kB
          Tsuyoshi Ozawa
        2. YARN-2313.1.patch
          5 kB
          Tsuyoshi Ozawa
        3. YARN-2313.2.patch
          5 kB
          Tsuyoshi Ozawa
        4. YARN-2313.3.patch
          5 kB
          Tsuyoshi Ozawa
        5. YARN-2313.4.patch
          6 kB
          Tsuyoshi Ozawa

          Activity

            People

            • Assignee:
              ozawa Tsuyoshi Ozawa
              Reporter:
              ozawa Tsuyoshi Ozawa
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: