Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2928 YARN Timeline Service v.2: alpha 1
  3. YARN-5045

hbase unit tests fail due to dependency issues

VotersWatch issueWatchersLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: YARN-2928
    • Fix Version/s: 2.9.0, 3.0.0-alpha1
    • Component/s: timelineserver
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After the 5/4 rebase, the hbase unit tests in the timeline service project are failing:

      org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage  Time elapsed: 5.103 sec  <<< ERROR!
      java.io.IOException: Shutting down
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:366)
      	at java.net.URLClassLoader$1.run(URLClassLoader.java:355)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at java.net.URLClassLoader.findClass(URLClassLoader.java:354)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:423)
      	at sun.misc.Launcher$AppClassLoader.loadClass(Launcher.java:308)
      	at java.lang.ClassLoader.loadClass(ClassLoader.java:356)
      	at org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
      	at org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
      	at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:500)
      	at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:104)
      	at org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
      	at org.apache.hadoop.hbase.http.InfoServer.<init>(InfoServer.java:77)
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
      	at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:550)
      	at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
      	at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
      	at org.apache.hadoop.hbase.LocalHBaseCluster.addMaster(LocalHBaseCluster.java:217)
      	at org.apache.hadoop.hbase.LocalHBaseCluster.<init>(LocalHBaseCluster.java:153)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.init(MiniHBaseCluster.java:213)
      	at org.apache.hadoop.hbase.MiniHBaseCluster.<init>(MiniHBaseCluster.java:93)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniHBaseCluster(HBaseTestingUtility.java:978)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:938)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:812)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:806)
      	at org.apache.hadoop.hbase.HBaseTestingUtility.startMiniCluster(HBaseTestingUtility.java:750)
      	at org.apache.hadoop.yarn.server.timelineservice.reader.TestTimelineReaderWebServicesHBaseStorage.setup(TestTimelineReaderWebServicesHBaseStorage.java:87)
      

      The root cause is that the hbase mini server depends on hadoop common's MetricsServlet which has been removed in the trunk (HADOOP-12504):

      Caused by: java.lang.NoClassDefFoundError: org/apache/hadoop/metrics/MetricsServlet
              at org.apache.hadoop.hbase.http.HttpServer.addDefaultServlets(HttpServer.java:677)
              at org.apache.hadoop.hbase.http.HttpServer.initializeWebServer(HttpServer.java:546)
              at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:500)
              at org.apache.hadoop.hbase.http.HttpServer.<init>(HttpServer.java:104)
              at org.apache.hadoop.hbase.http.HttpServer$Builder.build(HttpServer.java:345)
              at org.apache.hadoop.hbase.http.InfoServer.<init>(InfoServer.java:77)
              at org.apache.hadoop.hbase.regionserver.HRegionServer.putUpWebUI(HRegionServer.java:1697)
              at org.apache.hadoop.hbase.regionserver.HRegionServer.<init>(HRegionServer.java:550)
              at org.apache.hadoop.hbase.master.HMaster.<init>(HMaster.java:333)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:525)
              at org.apache.hadoop.hbase.util.JVMClusterUtil.createMasterThread(JVMClusterUtil.java:139)
              ... 26 more
      

        Attachments

        1. YARN-5045-YARN-2928.01.patch
          24 kB
          Sangjin Lee
        2. YARN-5045-YARN-2928.02.patch
          25 kB
          Sangjin Lee
        3. YARN-5045-YARN-2928.03.patch
          26 kB
          Sangjin Lee
        4. YARN-5045-YARN-2928.poc.patch
          24 kB
          Sangjin Lee

        Issue Links

          Activity

            People

            • Assignee:
              sjlee0 Sangjin Lee
              Reporter:
              sjlee0 Sangjin Lee

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment