Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12773

HBase classes fail to load with client/job classloader enabled

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 2.6.3
    • Fix Version/s: 2.8.0, 2.7.3, 2.6.5, 3.0.0-alpha1
    • Component/s: util
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      Currently if a user uses HBase and enables the client/job classloader, the job fails to load HBase classes. For example,

      java.lang.NoClassDefFoundError: Lorg/apache/hadoop/hbase/client/HBaseAdmin;
      	at java.lang.Class.getDeclaredFields0(Native Method)
      	at java.lang.Class.privateGetDeclaredFields(Class.java:2509)
      	at java.lang.Class.getDeclaredField(Class.java:1959)
      	at java.io.ObjectStreamClass.getDeclaredSUID(ObjectStreamClass.java:1703)
      	at java.io.ObjectStreamClass.access$700(ObjectStreamClass.java:72)
      	at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:484)
      	at java.io.ObjectStreamClass$2.run(ObjectStreamClass.java:472)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.io.ObjectStreamClass.<init>(ObjectStreamClass.java:472)
      	at java.io.ObjectStreamClass.lookup(ObjectStreamClass.java:369)
      

      It is because the HBase classes (org.apache.hadoop.hbase.*) meet the system classes criteria which are supposed to be loaded strictly from the base classloader. But hadoop does not provide HBase as a dependency.

      We should exclude the HBase classes from the system classes until/unless HBase is provided by a future version of hadoop.

        Activity

        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 14s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 16s Maven dependency ordering for branch
        +1 mvninstall 9m 18s trunk passed
        +1 compile 11m 12s trunk passed with JDK v1.8.0_66
        +1 compile 9m 4s trunk passed with JDK v1.7.0_91
        +1 mvnsite 1m 16s trunk passed
        +1 mvneclipse 0m 15s trunk passed
        +1 javadoc 1m 12s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 18s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 8s Maven dependency ordering for patch
        +1 mvninstall 0m 51s the patch passed
        +1 compile 10m 44s the patch passed with JDK v1.8.0_66
        +1 javac 10m 44s the patch passed
        +1 compile 7m 59s the patch passed with JDK v1.7.0_91
        +1 javac 7m 59s the patch passed
        +1 mvnsite 1m 3s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 javadoc 0m 57s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 5s the patch passed with JDK v1.7.0_91
        +1 unit 8m 13s hadoop-common in the patch passed with JDK v1.8.0_66.
        +1 unit 8m 1s hadoop-common in the patch passed with JDK v1.7.0_91.
        +1 asflicense 0m 22s Patch does not generate ASF License warnings.
        74m 42s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786570/HADOOP-12773.01.patch
        JIRA Issue HADOOP-12773
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit
        uname Linux 69d308514204 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / a37e423
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8553/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 77MB
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8553/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 16s Maven dependency ordering for branch +1 mvninstall 9m 18s trunk passed +1 compile 11m 12s trunk passed with JDK v1.8.0_66 +1 compile 9m 4s trunk passed with JDK v1.7.0_91 +1 mvnsite 1m 16s trunk passed +1 mvneclipse 0m 15s trunk passed +1 javadoc 1m 12s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 18s trunk passed with JDK v1.7.0_91 0 mvndep 0m 8s Maven dependency ordering for patch +1 mvninstall 0m 51s the patch passed +1 compile 10m 44s the patch passed with JDK v1.8.0_66 +1 javac 10m 44s the patch passed +1 compile 7m 59s the patch passed with JDK v1.7.0_91 +1 javac 7m 59s the patch passed +1 mvnsite 1m 3s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 javadoc 0m 57s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 5s the patch passed with JDK v1.7.0_91 +1 unit 8m 13s hadoop-common in the patch passed with JDK v1.8.0_66. +1 unit 8m 1s hadoop-common in the patch passed with JDK v1.7.0_91. +1 asflicense 0m 22s Patch does not generate ASF License warnings. 74m 42s Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786570/HADOOP-12773.01.patch JIRA Issue HADOOP-12773 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit uname Linux 69d308514204 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / a37e423 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8553/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 77MB Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8553/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 10s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
        0 mvndep 0m 8s Maven dependency ordering for branch
        +1 mvninstall 7m 17s trunk passed
        +1 compile 7m 9s trunk passed with JDK v1.8.0_66
        +1 compile 7m 21s trunk passed with JDK v1.7.0_91
        +1 mvnsite 1m 9s trunk passed
        +1 mvneclipse 0m 14s trunk passed
        +1 javadoc 0m 59s trunk passed with JDK v1.8.0_66
        +1 javadoc 1m 8s trunk passed with JDK v1.7.0_91
        0 mvndep 0m 9s Maven dependency ordering for patch
        +1 mvninstall 0m 47s the patch passed
        +1 compile 7m 9s the patch passed with JDK v1.8.0_66
        +1 javac 7m 9s the patch passed
        +1 compile 7m 32s the patch passed with JDK v1.7.0_91
        +1 javac 7m 32s the patch passed
        +1 mvnsite 1m 11s the patch passed
        +1 mvneclipse 0m 14s the patch passed
        +1 whitespace 0m 0s Patch has no whitespace issues.
        +1 javadoc 0m 59s the patch passed with JDK v1.8.0_66
        +1 javadoc 1m 10s the patch passed with JDK v1.7.0_91
        -1 unit 7m 59s hadoop-common in the patch failed with JDK v1.8.0_66.
        -1 unit 8m 11s hadoop-common in the patch failed with JDK v1.7.0_91.
        +1 asflicense 0m 24s Patch does not generate ASF License warnings.
        62m 17s



        Reason Tests
        JDK v1.8.0_66 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager
        JDK v1.7.0_91 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:0ca8df7
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786671/HADOOP-12773.01.patch
        JIRA Issue HADOOP-12773
        Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit
        uname Linux b8321ca79a5a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 29ae258
        Default Java 1.7.0_91
        Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt
        unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt
        JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/testReport/
        modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
        Max memory used 77MB
        Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/console
        Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. -1 test4tests 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. 0 mvndep 0m 8s Maven dependency ordering for branch +1 mvninstall 7m 17s trunk passed +1 compile 7m 9s trunk passed with JDK v1.8.0_66 +1 compile 7m 21s trunk passed with JDK v1.7.0_91 +1 mvnsite 1m 9s trunk passed +1 mvneclipse 0m 14s trunk passed +1 javadoc 0m 59s trunk passed with JDK v1.8.0_66 +1 javadoc 1m 8s trunk passed with JDK v1.7.0_91 0 mvndep 0m 9s Maven dependency ordering for patch +1 mvninstall 0m 47s the patch passed +1 compile 7m 9s the patch passed with JDK v1.8.0_66 +1 javac 7m 9s the patch passed +1 compile 7m 32s the patch passed with JDK v1.7.0_91 +1 javac 7m 32s the patch passed +1 mvnsite 1m 11s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 javadoc 0m 59s the patch passed with JDK v1.8.0_66 +1 javadoc 1m 10s the patch passed with JDK v1.7.0_91 -1 unit 7m 59s hadoop-common in the patch failed with JDK v1.8.0_66. -1 unit 8m 11s hadoop-common in the patch failed with JDK v1.7.0_91. +1 asflicense 0m 24s Patch does not generate ASF License warnings. 62m 17s Reason Tests JDK v1.8.0_66 Failed junit tests hadoop.security.ssl.TestReloadingX509TrustManager JDK v1.7.0_91 Failed junit tests hadoop.fs.shell.TestCopyPreserveFlag Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12786671/HADOOP-12773.01.patch JIRA Issue HADOOP-12773 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit uname Linux b8321ca79a5a 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 29ae258 Default Java 1.7.0_91 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_66 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_91 unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt unit https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt unit test logs https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.8.0_66.txt https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/artifact/patchprocess/patch-unit-hadoop-common-project_hadoop-common-jdk1.7.0_91.txt JDK v1.7.0_91 Test Results https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/testReport/ modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Max memory used 77MB Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/8559/console Powered by Apache Yetus 0.2.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kihwal Kihwal Lee added a comment -

        +1 makes sense.

        Show
        kihwal Kihwal Lee added a comment - +1 makes sense.
        Hide
        sjlee0 Sangjin Lee added a comment -

        Committed the patch. Thanks for your +1 Kihwal Lee!

        Show
        sjlee0 Sangjin Lee added a comment - Committed the patch. Thanks for your +1 Kihwal Lee !
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Hadoop-trunk-Commit #9263 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9263/)
        HADOOP-12773. HBase classes fail to load with client/job classloader (sjlee: rev 58acbf940a92ef8a761208a7a743175ee7b3377d)

        • hadoop-common-project/hadoop-common/src/main/resources/org.apache.hadoop.application-classloader.properties
        • hadoop-common-project/hadoop-common/CHANGES.txt
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9263 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9263/ ) HADOOP-12773 . HBase classes fail to load with client/job classloader (sjlee: rev 58acbf940a92ef8a761208a7a743175ee7b3377d) hadoop-common-project/hadoop-common/src/main/resources/org.apache.hadoop.application-classloader.properties hadoop-common-project/hadoop-common/CHANGES.txt
        Hide
        vinodkv Vinod Kumar Vavilapalli added a comment -

        Closing the JIRA as part of 2.7.3 release.

        Show
        vinodkv Vinod Kumar Vavilapalli added a comment - Closing the JIRA as part of 2.7.3 release.

          People

          • Assignee:
            sjlee0 Sangjin Lee
            Reporter:
            sjlee0 Sangjin Lee
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development