Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-301

Fair scheduler throws ConcurrentModificationException when iterating over app's priorities

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      In my test cluster, fairscheduler appear to concurrentModificationException and RM crash, here is the message:

      2012-12-30 17:14:17,171 FATAL org.apache.hadoop.yarn.server.resourcemanager.ResourceManager: Error in handling event type NODE_UPDATE to the scheduler
      java.util.ConcurrentModificationException
      at java.util.TreeMap$PrivateEntryIterator.nextEntry(TreeMap.java:1100)
      at java.util.TreeMap$KeyIterator.next(TreeMap.java:1154)
      at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.AppSchedulable.assignContainer(AppSchedulable.java:297)
      at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.assignContainer(FSLeafQueue.java:181)
      at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.nodeUpdate(FairScheduler.java:780)
      at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:842)
      at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:98)
      at org.apache.hadoop.yarn.server.resourcemanager.ResourceManager$SchedulerEventDispatcher$EventProcessor.run(ResourceManager.java:340)
      at java.lang.Thread.run(Thread.java:662)

      Attachments

        1. YARN-301.patch
          1 kB
          shenh062326
        2. YARN-301.patch
          4 kB
          shenh062326

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shenhong shenh062326
            shenhong shenh062326
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment