Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-22052

pom cleaning; filter out jersey-core in hadoop2 to match hadoop3 and remove redunant version specifications

    XMLWordPrintableJSON

Details

    • Task
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 3.0.0-alpha-1, 2.2.0, 2.0.5, 2.3.0, 2.1.4
    • None
    • None
    • Hide
      <!-- markdown -->
      Fixed awkward dependency issue that prevented site building.

      #### note specific to HBase 2.1.4
      HBase 2.1.4 shipped with an early version of this fix that incorrectly altered the libraries included in our binary assembly for using Apache Hadoop 2.7 (the current build default Hadoop version for 2.1.z). For folks running out of the box against a Hadoop 2.7 cluster (or folks who skip the installation step of [replacing the bundled Hadoop libraries](http://hbase.apache.org/book.html#hadoop)) this will result in a failure at Region Server startup due to a missing class definition. e.g.:
      ```
      2019-03-27 09:02:05,779 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer
      java.lang.NoClassDefFoundError: org/apache/htrace/SamplerBuilder
      at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:644)
      at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628)
      at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149)
      at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667)
      at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93)
      at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701)
      at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171)
      at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356)
      at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
      at org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:362)
      at org.apache.hadoop.hbase.util.CommonFSUtils.isValidWALRootDir(CommonFSUtils.java:411)
      at org.apache.hadoop.hbase.util.CommonFSUtils.getWALRootDir(CommonFSUtils.java:387)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:704)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:613)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:3029)
      at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:63)
      at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87)
      at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70)
      at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149)
      at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3047)
      Caused by: java.lang.ClassNotFoundException: org.apache.htrace.SamplerBuilder
      at java.net.URLClassLoader.findClass(URLClassLoader.java:381)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:424)
      at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349)
      at java.lang.ClassLoader.loadClass(ClassLoader.java:357)
      ... 26 more

      ```

      Workaround via any _one_ of the following:
      * If you are running against a Hadoop cluster that is 2.8+, ensure you replace the Hadoop libaries in the default binary assembly with those for your version.
      * If you are running against a Hadoop cluster that is 2.8+, build the binary assembly from the source release while specifying your Hadoop version.
      * If you are running against a Hadoop cluster that is a supported 2.7 release, ensure the `hadoop` executable is in the `PATH` seen at Region Server startup and that you are not using the `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP` bypass.
      * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers via the HBASE_CLASSPATH environment variable.
      * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers by copying it into the directory `${HBASE_HOME}/lib/client-facing-thirdparty/`.
      Show
      <!-- markdown --> Fixed awkward dependency issue that prevented site building. #### note specific to HBase 2.1.4 HBase 2.1.4 shipped with an early version of this fix that incorrectly altered the libraries included in our binary assembly for using Apache Hadoop 2.7 (the current build default Hadoop version for 2.1.z). For folks running out of the box against a Hadoop 2.7 cluster (or folks who skip the installation step of [replacing the bundled Hadoop libraries]( http://hbase.apache.org/book.html#hadoop) ) this will result in a failure at Region Server startup due to a missing class definition. e.g.: ``` 2019-03-27 09:02:05,779 ERROR [main] regionserver.HRegionServer: Failed construction RegionServer java.lang.NoClassDefFoundError: org/apache/htrace/SamplerBuilder at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:644) at org.apache.hadoop.hdfs.DFSClient.<init>(DFSClient.java:628) at org.apache.hadoop.hdfs.DistributedFileSystem.initialize(DistributedFileSystem.java:149) at org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2667) at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:93) at org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2701) at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2683) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:372) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:171) at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:356) at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295) at org.apache.hadoop.hbase.util.CommonFSUtils.getRootDir(CommonFSUtils.java:362) at org.apache.hadoop.hbase.util.CommonFSUtils.isValidWALRootDir(CommonFSUtils.java:411) at org.apache.hadoop.hbase.util.CommonFSUtils.getWALRootDir(CommonFSUtils.java:387) at org.apache.hadoop.hbase.regionserver.HRegionServer.initializeFileSystem(HRegionServer.java:704) at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:613) at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:423) at org.apache.hadoop.hbase.regionserver.HRegionServer.constructRegionServer(HRegionServer.java:3029) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.start(HRegionServerCommandLine.java:63) at org.apache.hadoop.hbase.regionserver.HRegionServerCommandLine.run(HRegionServerCommandLine.java:87) at org.apache.hadoop.util.ToolRunner.run(ToolRunner.java:70) at org.apache.hadoop.hbase.util.ServerCommandLine.doMain(ServerCommandLine.java:149) at org.apache.hadoop.hbase.regionserver.HRegionServer.main(HRegionServer.java:3047) Caused by: java.lang.ClassNotFoundException: org.apache.htrace.SamplerBuilder at java.net.URLClassLoader.findClass(URLClassLoader.java:381) at java.lang.ClassLoader.loadClass(ClassLoader.java:424) at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:349) at java.lang.ClassLoader.loadClass(ClassLoader.java:357) ... 26 more ``` Workaround via any _one_ of the following: * If you are running against a Hadoop cluster that is 2.8+, ensure you replace the Hadoop libaries in the default binary assembly with those for your version. * If you are running against a Hadoop cluster that is 2.8+, build the binary assembly from the source release while specifying your Hadoop version. * If you are running against a Hadoop cluster that is a supported 2.7 release, ensure the `hadoop` executable is in the `PATH` seen at Region Server startup and that you are not using the `HBASE_DISABLE_HADOOP_CLASSPATH_LOOKUP` bypass. * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers via the HBASE_CLASSPATH environment variable. * For any supported Hadoop version, manually make the Apache HTrace artifact `htrace-core-3.1.0-incubating.jar` available to all Region Servers by copying it into the directory `${HBASE_HOME}/lib/client-facing-thirdparty/`.

    Description

      Working on HBASE-22029, where we fail compile of hbase-it module four hours into an RC build, it looks like transitive include of jersey-core is the culprit. hadoop3 profile does a bunch of jersey-core exclusions. This issue is about having hadoop2 profile and hadoop3 profiles match around jersey-core treatment. Some miscellaneous cleanups are also done.

      Attachments

        1. HBASE-22052.branch-2.0.001.patch
          16 kB
          Michael Stack
        2. HBASE-22052.branch-2.0.002.patch
          15 kB
          Michael Stack
        3. HBASE-22052.branch-2.0.002.patch
          15 kB
          Michael Stack
        4. HBASE-22052.branch-2.0.003.patch
          17 kB
          Michael Stack
        5. HBASE-22052.branch-2.1.001.patch
          19 kB
          Michael Stack
        6. HBASE-22052.branch-2.0.004.patch
          39 kB
          Michael Stack
        7. HBASE-22052.branch-2.1.002.patch
          19 kB
          Michael Stack
        8. HBASE-22052.branch-2.2.001.patch
          18 kB
          Michael Stack
        9. HBASE-22052.branch-2.2.002.patch
          19 kB
          Michael Stack
        10. HBASE-22052.branch-2.1.003.patch
          20 kB
          Michael Stack
        11. HBASE-22052.2.1.003.addendum1.txt
          0.0 kB
          Michael Stack
        12. HBASE-22052-branch-2.1.003.addendum2.txt
          2 kB
          Michael Stack
        13. nothing_change.patch
          2 kB
          Michael Stack
        14. nothing_change.patch
          2 kB
          Michael Stack

        Issue Links

          Activity

            People

              stack Michael Stack
              stack Michael Stack
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: