Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: build, client
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed

      Description

      MAPREDUCE-2337 deprecated getJobTrackerState() on ClusterStatus, this issue is to remove the getter (in favour of getJobTrackerStatus(), which will remain) so there is no longer a direct dependency of the public API on JobTracker. This is for MAPREDUCE-1638.

        Activity

        Hide
        Tom White added a comment -

        Here's a patch for this change.

        Show
        Tom White added a comment - Here's a patch for this change.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12477443/MAPREDUCE-2455.patch
        against trunk revision 1096649.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 18 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477443/MAPREDUCE-2455.patch against trunk revision 1096649. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 18 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/187//console This message is automatically generated.
        Hide
        Todd Lipcon added a comment -

        Since this was just deprecated in trunk, not 0.20, can we actually remove it yet?

        Show
        Todd Lipcon added a comment - Since this was just deprecated in trunk, not 0.20, can we actually remove it yet?
        Hide
        Tom White added a comment -

        The deprecation (MAPREDUCE-2337) will go into 0.22, and then I was hoping this could go into 0.23. Removing this method allows the division of API and implementation in MAPREDUCE-1638, which will be beneficial for MAPREDUCE-279, for example (which is targetted for 0.23, as I understand it).

        Show
        Tom White added a comment - The deprecation ( MAPREDUCE-2337 ) will go into 0.22, and then I was hoping this could go into 0.23. Removing this method allows the division of API and implementation in MAPREDUCE-1638 , which will be beneficial for MAPREDUCE-279 , for example (which is targetted for 0.23, as I understand it).
        Hide
        Todd Lipcon added a comment -

        Tom and I discussed this a bit. Although it's not clear whether we can remove everything deprecated in 0.22 yet, we think it's OK to remove this one for the following rationale:

        • the API isn't commonly used (eg Pig doesn't use it at all and Hive uses it in just one place)
        • the workaround to deal with the incompatibility is a trivial rename
        • this is blocking MAPREDUCE-1638 which we really need to do sooner than later

        So, +1

        Show
        Todd Lipcon added a comment - Tom and I discussed this a bit. Although it's not clear whether we can remove everything deprecated in 0.22 yet, we think it's OK to remove this one for the following rationale: the API isn't commonly used (eg Pig doesn't use it at all and Hive uses it in just one place) the workaround to deal with the incompatibility is a trivial rename this is blocking MAPREDUCE-1638 which we really need to do sooner than later So, +1
        Hide
        Arun C Murthy added a comment -

        I agree that we shud fix this for 23.

        In fact, I'd go further and actually consider removing this api. The JT is always in RUNNING state (while not dead ) and thus I don't see the utility of this api. Maybe Hive really doesn't need it?

        Show
        Arun C Murthy added a comment - I agree that we shud fix this for 23. In fact, I'd go further and actually consider removing this api. The JT is always in RUNNING state (while not dead ) and thus I don't see the utility of this api. Maybe Hive really doesn't need it?
        Hide
        Tom White added a comment -

        > In fact, I'd go further and actually consider removing this api.

        I agree with the sentiment. ClusterStatus shouldn't be used as a monitoring API, since metrics (or other monitoring systems) should be used for that purpose. In particular, the API that ClusterStatus (or ClusterMetrics in the new API) exposes doesn't fit with MAPREDUCE-279. It's premature to remove them in 0.23, but it makes sense to consider replacements for them.

        Of course, we still need this JIRA to move forward with MAPREDUCE-1638 and the code organization for MAPREDUCE-279.

        Show
        Tom White added a comment - > In fact, I'd go further and actually consider removing this api. I agree with the sentiment. ClusterStatus shouldn't be used as a monitoring API, since metrics (or other monitoring systems) should be used for that purpose. In particular, the API that ClusterStatus (or ClusterMetrics in the new API) exposes doesn't fit with MAPREDUCE-279 . It's premature to remove them in 0.23, but it makes sense to consider replacements for them. Of course, we still need this JIRA to move forward with MAPREDUCE-1638 and the code organization for MAPREDUCE-279 .
        Hide
        Arun C Murthy added a comment -

        +1 for doing this presently.

        Show
        Arun C Murthy added a comment - +1 for doing this presently.
        Hide
        Tom White added a comment -

        I've just committed this.

        Show
        Tom White added a comment - I've just committed this.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #706 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/706/)
        MAPREDUCE-2455. Remove deprecated JobTracker.State in favour of JobTrackerStatus.

        tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1129771
        Files :

        • /hadoop/mapreduce/trunk/src/contrib/capacity-scheduler/src/test/org/apache/hadoop/mapred/CapacityTestUtils.java
        • /hadoop/mapreduce/trunk/CHANGES.txt
        • /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/FakeObjectUtilities.java
        • /hadoop/mapreduce/trunk/src/contrib/fairscheduler/src/test/org/apache/hadoop/mapred/TestFairScheduler.java
        • /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/ClusterStatus.java
        • /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestJobQueueTaskScheduler.java
        • /hadoop/mapreduce/trunk/src/webapps/job/jobtracker.jspx
        • /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestParallelInitialization.java
        • /hadoop/mapreduce/trunk/src/webapps/job/jobtracker.jsp
        • /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestJobInProgress.java
        • /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/JobTracker.java
        • /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/JobClient.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #706 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/706/ ) MAPREDUCE-2455 . Remove deprecated JobTracker.State in favour of JobTrackerStatus. tomwhite : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1129771 Files : /hadoop/mapreduce/trunk/src/contrib/capacity-scheduler/src/test/org/apache/hadoop/mapred/CapacityTestUtils.java /hadoop/mapreduce/trunk/CHANGES.txt /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/FakeObjectUtilities.java /hadoop/mapreduce/trunk/src/contrib/fairscheduler/src/test/org/apache/hadoop/mapred/TestFairScheduler.java /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/ClusterStatus.java /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestJobQueueTaskScheduler.java /hadoop/mapreduce/trunk/src/webapps/job/jobtracker.jspx /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestParallelInitialization.java /hadoop/mapreduce/trunk/src/webapps/job/jobtracker.jsp /hadoop/mapreduce/trunk/src/test/mapred/org/apache/hadoop/mapred/TestJobInProgress.java /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/JobTracker.java /hadoop/mapreduce/trunk/src/java/org/apache/hadoop/mapred/JobClient.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #722 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/722/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #722 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/722/ )

          People

          • Assignee:
            Tom White
            Reporter:
            Tom White
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development