Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-3843

Fair Scheduler should not accept apps with space keys as queue name

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Duplicate
    • Affects Version/s: 2.4.0, 2.5.0
    • Fix Version/s: None
    • Component/s: fairscheduler
    • Labels:
      None
    • Target Version/s:

      Description

      As YARN-461, since empty string queue name is not valid, queue name with space keys such as " " ," " should not be accepted either, also not as prefix nor postfix.
      e.g) "root.test.queuename ", or "root.test. queuename"

      I have 2 specific cases kill RM with these space keys as part of queue name.
      1) Without placement policy (hadoop 2.4.0 and above),
      When a job is submitted with " "(space key) as queue name
      e.g) mapreduce.job.queuename=" "

      2) With placement policy (hadoop 2.5.0 and above)
      Once a job is submitted without space key as queue name, and submit another job with space key.
      e.g) 1st time: mapreduce.job.queuename="root.test.user1"
      2nd time: mapreduce.job.queuename="root.test.user1 "

      Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 0.974 sec <<< FAILURE! - in org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler
      testQueueNameWithSpace(org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler)  Time elapsed: 0.724 sec  <<< ERROR!
      org.apache.hadoop.metrics2.MetricsException: Metrics source QueueMetrics,q0=root,q1=adhoc,q2=birvine already exists!
      	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.newSourceName(DefaultMetricsSystem.java:135)
      	at org.apache.hadoop.metrics2.lib.DefaultMetricsSystem.sourceName(DefaultMetricsSystem.java:112)
      	at org.apache.hadoop.metrics2.impl.MetricsSystemImpl.register(MetricsSystemImpl.java:218)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueueMetrics.forQueue(FSQueueMetrics.java:96)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSQueue.<init>(FSQueue.java:56)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FSLeafQueue.<init>(FSLeafQueue.java:66)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.createQueue(QueueManager.java:169)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getQueue(QueueManager.java:120)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.QueueManager.getLeafQueue(QueueManager.java:88)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.assignToQueue(FairScheduler.java:660)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.addApplication(FairScheduler.java:569)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.FairScheduler.handle(FairScheduler.java:1127)
      	at org.apache.hadoop.yarn.server.resourcemanager.scheduler.fair.TestFairScheduler.testQueueNameWithSpace(TestFairScheduler.java:627)
      

        Attachments

        1. YARN-3843.01.patch
          6 kB
          Dongwook Kwon

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                dongwook Dongwook Kwon
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: