Hadoop Common
  1. Hadoop Common
  2. HADOOP-6951

Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.22.0
    • Component/s: security
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Because the protocol -> ACL mapping in ServiceAuthorizationManager is static, services which are run in the same JVM have the potential to clobber the other's service authorization ACLs whenever ServiceAuthorizationManager.refresh() is called. This causes authorization failures if one tries to launch a 2NN connected to a minicluster with hadoop.security.authorization enabled. Seems like each service should have its own instance of a ServiceAuthorizationManager, instead of using static methods.

      1. hadoop-6951.1.txt
        5 kB
        Aaron T. Myers
      2. hadoop-6951.2.txt
        6 kB
        Aaron T. Myers
      3. hadoop-6951.txt.0
        5 kB
        Aaron T. Myers

        Issue Links

          Activity

          Hide
          Aaron T. Myers added a comment -

          Change ServiceAuthorizationManager to have a non-static map, and change Server to include an instance of ServiceAuthorizationManager.

          Show
          Aaron T. Myers added a comment - Change ServiceAuthorizationManager to have a non-static map, and change Server to include an instance of ServiceAuthorizationManager.
          Hide
          Todd Lipcon added a comment -

          Looks pretty good. One small thing - rather than making serviceAuthorizationManager public in Server, can you add a public getter marked only used for tests?

          Show
          Todd Lipcon added a comment - Looks pretty good. One small thing - rather than making serviceAuthorizationManager public in Server, can you add a public getter marked only used for tests?
          Hide
          Aaron T. Myers added a comment -

          Updated patch to address Todd's comments.

          Show
          Aaron T. Myers added a comment - Updated patch to address Todd's comments.
          Hide
          Tom White added a comment -

          This seems like a good approach. +1

          You could mark getServiceAuthorizationManager() on Server as {{@InterfaceAudience.LimitedPrivate(

          {"HDFS", "MapReduce"}

          )}} since it is only needed by tests.

          Show
          Tom White added a comment - This seems like a good approach. +1 You could mark getServiceAuthorizationManager() on Server as {{@InterfaceAudience.LimitedPrivate( {"HDFS", "MapReduce"} )}} since it is only needed by tests.
          Hide
          Aaron T. Myers added a comment -

          Thanks for the review, Tom.

          Updated patch to address Tom's comments.

          Show
          Aaron T. Myers added a comment - Thanks for the review, Tom. Updated patch to address Tom's comments.
          Hide
          Aaron T. Myers added a comment -

          I just ran the tests, with the following results. The javadoc warning was present before my patch.

          -1 overall.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 3 new or modified tests.

          -1 javadoc. The javadoc tool appears to have generated 1 warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 system tests framework. The patch passed system tests framework compile.

          Show
          Aaron T. Myers added a comment - I just ran the tests, with the following results. The javadoc warning was present before my patch. -1 overall. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 javadoc. The javadoc tool appears to have generated 1 warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 system tests framework. The patch passed system tests framework compile.
          Hide
          Tom White added a comment -

          I'd like to commit this in the next day or so unless there are any objections.

          Show
          Tom White added a comment - I'd like to commit this in the next day or so unless there are any objections.
          Hide
          Tom White added a comment -

          I've just committed this. Thanks, Aaron!

          (I checked that the unit tests passed before committing this.)

          Show
          Tom White added a comment - I've just committed this. Thanks, Aaron! (I checked that the unit tests passed before committing this.)
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #378 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/378/)
          HADOOP-6951. Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies. Contributed by Aaron T. Myers

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #378 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/378/ ) HADOOP-6951 . Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies. Contributed by Aaron T. Myers
          Hide
          Suresh Srinivas added a comment -

          This code in common breaks HDFS. Currently HDFS trunk does not compile! HDFS-1422 has been files for this. Please close it, if it is not necessary.

          Show
          Suresh Srinivas added a comment - This code in common breaks HDFS. Currently HDFS trunk does not compile! HDFS-1422 has been files for this. Please close it, if it is not necessary.
          Hide
          Tom White added a comment -

          Sorry about this, I overlooked the fact that the HDFS and MR issues needed committing at the same time. We should temporarily revert this (the related issues need unit tests and test-patch running). Unfortunately, Apache svn is not responding for me at the moment.

          Show
          Tom White added a comment - Sorry about this, I overlooked the fact that the HDFS and MR issues needed committing at the same time. We should temporarily revert this (the related issues need unit tests and test-patch running). Unfortunately, Apache svn is not responding for me at the moment.
          Hide
          Tom White added a comment -

          I've now reverted this in trunk.

          Show
          Tom White added a comment - I've now reverted this in trunk.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #379 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/379/)
          Reverting commit 1001067 while related HDFS and MR JIRAs are tested (HADOOP-6951).

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #379 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/379/ ) Reverting commit 1001067 while related HDFS and MR JIRAs are tested ( HADOOP-6951 ).
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #461 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/461/)
          Reverting commit 1001067 while related HDFS and MR JIRAs are tested (HADOOP-6951).
          HADOOP-6951. Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies. Contributed by Aaron T. Myers

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #461 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/461/ ) Reverting commit 1001067 while related HDFS and MR JIRAs are tested ( HADOOP-6951 ). HADOOP-6951 . Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies. Contributed by Aaron T. Myers
          Hide
          Aaron T. Myers added a comment -

          Hi Tom,

          I've now run test-core and test-patch on both HDFS-1399 and MAPREDUCE-2067. Would you mind taking another look at this?

          Thanks a lot,
          Aaron

          Show
          Aaron T. Myers added a comment - Hi Tom, I've now run test-core and test-patch on both HDFS-1399 and MAPREDUCE-2067 . Would you mind taking another look at this? Thanks a lot, Aaron
          Hide
          Tom White added a comment -

          I've just committed this (again). Thanks Aaron!

          I'll commit the other two JIRAs shortly.

          Show
          Tom White added a comment - I've just committed this (again). Thanks Aaron! I'll commit the other two JIRAs shortly.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #381 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/381/)
          HADOOP-6951. Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies. Contributed by Aaron T. Myers.

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #381 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk-Commit/381/ ) HADOOP-6951 . Distinct minicluster services (e.g. NN and JT) overwrite each other's service policies. Contributed by Aaron T. Myers.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk #489 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/489/)

          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk #489 (See https://hudson.apache.org/hudson/job/Hadoop-Common-trunk/489/ )

            People

            • Assignee:
              Aaron T. Myers
              Reporter:
              Aaron T. Myers
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development