Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2979

Remove ClientProtocolProvider configuration under mapreduce-client-core

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha
    • Fix Version/s: 0.23.0
    • Component/s: mrv2
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      ClientProtocolProvider configuration exists under the job-client and core modules. It's really only required in job-client. The version in core points to JobTrackerClientProtocolProvider which causes

      java.util.ServiceConfigurationError: org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider: Provider org.apache.hadoop.mapred.JobTrackerClientProtocolProvider not found
      at java.util.ServiceLoader.fail(ServiceLoader.java:214)
      at java.util.ServiceLoader.access$400(ServiceLoader.java:164)
      at java.util.ServiceLoader$LazyIterator.next(ServiceLoader.java:350)
      at java.util.ServiceLoader$1.next(ServiceLoader.java:421)
      at org.apache.hadoop.mapreduce.Cluster.<init>(Cluster.java:73)
      at org.apache.hadoop.mapreduce.Job.<init>(Job.java:133)
      at org.apache.hadoop.mapreduce.Job.<init>(Job.java:138)
      at org.apache.hadoop.examples.WordCount.main(WordCount.java:75)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.ProgramDriver$ProgramDescription.invoke(ProgramDriver.java:72)
      at org.apache.hadoop.util.ProgramDriver.driver(ProgramDriver.java:144)
      at org.apache.hadoop.examples.ExampleDriver.main(ExampleDriver.java:68)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:189)

      1. MR2979_v1.patch
        2 kB
        Amareshwari Sriramadasu
      2. MR2979_v1.patch
        2 kB
        Siddharth Seth

        Activity

        Hide
        Siddharth Seth added a comment -

        Removes a copy of the ClientProviderConfiguration under mr-client-core.
        Also post MR2864, added back the equivalent of 'yarn.apps.stagingDir' to mapred-default.xml (earlier yarn-dfault.xml)
        Trivial patch... doesn't require explicit tests.

        Show
        Siddharth Seth added a comment - Removes a copy of the ClientProviderConfiguration under mr-client-core. Also post MR2864, added back the equivalent of 'yarn.apps.stagingDir' to mapred-default.xml (earlier yarn-dfault.xml) Trivial patch... doesn't require explicit tests.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12493860/MR2979_v1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12493860/MR2979_v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-hs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-shuffle.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-app.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-jobclient.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-mapreduce-client-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-resourcemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//artifact/trunk/patchprocess/newPatchFindbugsWarningshadoop-yarn-api.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/686//console This message is automatically generated.
        Hide
        Mahadev konar added a comment -

        sid,
        Can you please confirm if ant tests are still working with this change? The changes look good to me.

        Show
        Mahadev konar added a comment - sid, Can you please confirm if ant tests are still working with this change? The changes look good to me.
        Hide
        Amareshwari Sriramadasu added a comment -

        Resubmitting to test MAPREDUCE-2978

        Show
        Amareshwari Sriramadasu added a comment - Resubmitting to test MAPREDUCE-2978
        Hide
        Amareshwari Sriramadasu added a comment -

        Reattaching the same patch

        Show
        Amareshwari Sriramadasu added a comment - Reattaching the same patch
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12494036/MR2979_v1.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in .

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/705//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/705//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/705//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12494036/MR2979_v1.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in . +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/705//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/705//artifact/trunk/hadoop-mapreduce-project/patchprocess/newPatchFindbugsWarningshadoop-yarn-server-nodemanager.html Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/705//console This message is automatically generated.
        Hide
        Vinod Kumar Vavilapalli added a comment -

        Karam Singh ran into the same error above today with one of his non-regular hadoop commands. We spent some time but are still baffled as to why we never ran into this before but hitting it only now.

        Regardless this patch should prevent the exception above.

        Looked at the patch. +1.

        Show
        Vinod Kumar Vavilapalli added a comment - Karam Singh ran into the same error above today with one of his non-regular hadoop commands. We spent some time but are still baffled as to why we never ran into this before but hitting it only now. Regardless this patch should prevent the exception above. Looked at the patch. +1.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #872 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/872/)
        MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #872 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/872/ ) MAPREDUCE-2979 . Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #949 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/949/)
        MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #949 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/949/ ) MAPREDUCE-2979 . Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Vinod Kumar Vavilapalli added a comment -

        -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

        Not related to the patch. But I am going to hunt the culprit down, and pester him/her all the way till he/she fixes it

        And oh, BTW, I just committed this trunk and branch 0.23. Thanks Sid!

        Show
        Vinod Kumar Vavilapalli added a comment - -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. Not related to the patch. But I am going to hunt the culprit down, and pester him/her all the way till he/she fixes it And oh, BTW, I just committed this trunk and branch 0.23. Thanks Sid!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #884 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/884/)
        MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #884 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/884/ ) MAPREDUCE-2979 . Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #816 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/816/)
        MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #816 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/816/ ) MAPREDUCE-2979 . Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk #792 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/792/)
        MAPREDUCE-2979. Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth.

        vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #792 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/792/ ) MAPREDUCE-2979 . Removed the needless ClientProtocolProvider configuration from the hadoop-mapreduce-client-core module. Contributed by Siddharth Seth. vinodkv : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1169808 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/META-INF/services/org.apache.hadoop.mapreduce.protocol.ClientProtocolProvider /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-core/src/main/resources/mapred-default.xml

          People

          • Assignee:
            Siddharth Seth
            Reporter:
            Siddharth Seth
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development