Hadoop Common
  1. Hadoop Common
  2. HADOOP-5485

Authorisation machanism required for acceesing jobtracker url :- jobtracker.com:port/scheduler

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      New Fair Scheduler configuration parameter webinterface.private.actions controls whether changes to pools and priorities are permitted from the web interface. Changes are not permitted by default.

      Description

      FS scheduler should have some mechanism to authorize people who can access the advanced scheduler url http://jobtracker.com:port/scheduler . In large clusters , which has hundreds of users, any user can access the url now and change the priority of his/her runing job. We don't want the users to change the job priority. So we should restrcit users accessing the link and only admins should have access to the link.

      1. HADOOP-5485.txt
        5 kB
        Vinod Kumar Vavilapalli
      2. HADOOP-5485.1.txt
        6 kB
        Vinod Kumar Vavilapalli

        Issue Links

          Activity

          Hide
          Matei Zaharia added a comment -

          As another option, we can have a config parameter for disabling updates through this UI. Admins can still change job priorities through the standard command line.

          Show
          Matei Zaharia added a comment - As another option, we can have a config parameter for disabling updates through this UI. Admins can still change job priorities through the standard command line.
          Hide
          Hemanth Yamijala added a comment -

          There are two approaches I can think of.

          In the short term, I think webinterface.private.actions should be checked and actions disabled in the web page. This is what the standard Hadoop JSPs do.
          In the longer term, we may want to unify the UI as per HADOOP-4712.

          Matei, the admin changes to job priorities via command line can be controlled by setting up ACLs, and is the way we are using here.

          Show
          Hemanth Yamijala added a comment - There are two approaches I can think of. In the short term, I think webinterface.private.actions should be checked and actions disabled in the web page. This is what the standard Hadoop JSPs do. In the longer term, we may want to unify the UI as per HADOOP-4712 . Matei, the admin changes to job priorities via command line can be controlled by setting up ACLs, and is the way we are using here.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Attaching patch to use webinterface.private.actions for disabling changes to priority/pool from the scheduler servlet. The patch also disables changes to the scheduling mode from the UI. The only nit is that this makes it impossible to change scheduling mode even by admins if webinterface.private.actions is disabled.

          Show
          Vinod Kumar Vavilapalli added a comment - Attaching patch to use webinterface.private.actions for disabling changes to priority/pool from the scheduler servlet. The patch also disables changes to the scheduling mode from the UI. The only nit is that this makes it impossible to change scheduling mode even by admins if webinterface.private.actions is disabled.
          Hide
          rahul k singh added a comment -

          looked at the code and web ui.
          +1

          Show
          rahul k singh added a comment - looked at the code and web ui. +1
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Pushing to Hudson.

          Matei, do the changes look ok to you? In summary, this patch uses webinterface.private.actions for 1) disabling changes to priority/pool from the scheduler servlet and 2) disabling the feature of changing scheduling mode(fair-share to FIFO and vice-versa) from the UI.

          Show
          Vinod Kumar Vavilapalli added a comment - Pushing to Hudson. Matei, do the changes look ok to you? In summary, this patch uses webinterface.private.actions for 1) disabling changes to priority/pool from the scheduler servlet and 2) disabling the feature of changing scheduling mode(fair-share to FIFO and vice-versa) from the UI.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12403425/HADOOP-5485.txt
          against trunk revision 762216.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12403425/HADOOP-5485.txt against trunk revision 762216. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/155/console This message is automatically generated.
          Hide
          Hemanth Yamijala added a comment -

          The changes look fine. One minor nit is that a new configuration object is being created here, which i think is unnecessary. I have two solutions:

          • We can use the conf object via the JSPUtil class - we could even have a arePrivateActionsAllowed kind of method maybe.. or just a simple accessor for the conf
          • The other option is to read this particular value, cache it and discard the conf.
          Show
          Hemanth Yamijala added a comment - The changes look fine. One minor nit is that a new configuration object is being created here, which i think is unnecessary. I have two solutions: We can use the conf object via the JSPUtil class - we could even have a arePrivateActionsAllowed kind of method maybe.. or just a simple accessor for the conf The other option is to read this particular value, cache it and discard the conf.
          Hide
          Vinod Kumar Vavilapalli added a comment -

          Attaching a new patch incorporating the above comments. Added a new method in JSPUtil

          boolean privateActionsAllowed()
          Show
          Vinod Kumar Vavilapalli added a comment - Attaching a new patch incorporating the above comments. Added a new method in JSPUtil boolean privateActionsAllowed()
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12405053/HADOOP-5485.1.txt
          against trunk revision 763502.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no tests are needed for this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed core unit tests.

          -1 contrib tests. The patch failed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12405053/HADOOP-5485.1.txt against trunk revision 763502. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed core unit tests. -1 contrib tests. The patch failed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-vesta.apache.org/172/console This message is automatically generated.
          Hide
          Hemanth Yamijala added a comment -

          Code changes look fine to me. Can you please verify the test failures are unrelated ? Unfortunately, I can't get the results from the link mentioned above. This may mean we need to run the tests manually.

          Show
          Hemanth Yamijala added a comment - Code changes look fine to me. Can you please verify the test failures are unrelated ? Unfortunately, I can't get the results from the link mentioned above. This may mean we need to run the tests manually.
          Hide
          Hemanth Yamijala added a comment -

          Actually, never mind. Searching through references of JspUtil, I found that all references are only in the new code or in existing JSPs which are not tested right now. So, I will hazard a guess that the test failures are unrelated and commit.

          Show
          Hemanth Yamijala added a comment - Actually, never mind. Searching through references of JspUtil, I found that all references are only in the new code or in existing JSPs which are not tested right now. So, I will hazard a guess that the test failures are unrelated and commit.
          Hide
          Hemanth Yamijala added a comment -

          I just committed this. Thanks, Vinod !

          Note that since the default value of webinterface.private.actions is false, this might appear like an incompatible interface change for the fair-scheduler servlet UI. But setting this value to true in the cluster's mapred-site.xml will get back the original behavior.

          Show
          Hemanth Yamijala added a comment - I just committed this. Thanks, Vinod ! Note that since the default value of webinterface.private.actions is false, this might appear like an incompatible interface change for the fair-scheduler servlet UI. But setting this value to true in the cluster's mapred-site.xml will get back the original behavior.
          Hide
          Hemanth Yamijala added a comment -

          Marking this an incompatible change, so that it gets attention. Also, setting webinterface.private.actions has other effects like enabling the 'kill job' and 'kill task' links on the Job web UI.

          Show
          Hemanth Yamijala added a comment - Marking this an incompatible change, so that it gets attention. Also, setting webinterface.private.actions has other effects like enabling the 'kill job' and 'kill task' links on the Job web UI.
          Hide
          Hudson added a comment -
          Show
          Hudson added a comment - Integrated in Hadoop-trunk #811 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/811/ )
          Hide
          Robert Chansler added a comment -

          Editorial pass over all release notes prior to publication of 0.21.

          Show
          Robert Chansler added a comment - Editorial pass over all release notes prior to publication of 0.21.

            People

            • Assignee:
              Vinod Kumar Vavilapalli
              Reporter:
              Aroop Maliakkal
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development