Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2630

MR-279: refreshQueues leads to NPEs when used w/FifoScheduler

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None
    • Environment:

      All

    • Hadoop Flags:
      Reviewed

      Description

      The RM's admin service exposes a method refreshQueues that is used to update the queue configuration when used with the CapacityScheduler, but if it is used with the FifoScheduler, it will set the containerTokenSecretManager/clusterTracker fields on the FifoScheduler to null, which eventually leads to NPE. Since the FifoScheduler only has one queue that cannot be refreshed, the correct behavior is for the refreshQueues call to be a no-op.

      I will attach a patch that fixes this by splitting the ResourceScheduler's reinitialize method into separate initialize/updateQueues methods.

      1. MAPREDUCE-2630.patch
        12 kB
        Josh Wills
      2. MAPREDUCE-2630-simple.patch
        2 kB
        Josh Wills

        Issue Links

          Activity

          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Mahadev konar made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Fix Version/s 0.23.0 [ 12315570 ]
          Resolution Fixed [ 1 ]
          Josh Wills made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Josh Wills made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Josh Wills made changes -
          Attachment MAPREDUCE-2630-simple.patch [ 12485614 ]
          Josh Wills made changes -
          Attachment MAPREDUCE-2630.patch [ 12485347 ]
          Josh Wills made changes -
          Attachment MAPREDUCE-2630.patch [ 12484600 ]
          Arun C Murthy made changes -
          Assignee Josh Wills [ jwills ]
          Jeff Hammerbacher made changes -
          Link This issue is related to MAPREDUCE-279 [ MAPREDUCE-279 ]
          Josh Wills made changes -
          Field Original Value New Value
          Attachment MAPREDUCE-2630.patch [ 12484600 ]
          Josh Wills created issue -

            People

            • Assignee:
              Josh Wills
              Reporter:
              Josh Wills
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Time Tracking

                Estimated:
                Original Estimate - 1h
                1h
                Remaining:
                Remaining Estimate - 1h
                1h
                Logged:
                Time Spent - Not Specified
                Not Specified

                  Development