Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-8246 Java 11/17 Support and Testing
  3. HDDS-8250

[JDK17] Recon crashes on startup with JDK 17

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Works for Me
    • None
    • None
    • Ozone Recon
    • None

    Description

      Instantly reproducible with ./compose/ozone docker-compose.yml with JDK 17 image (follow HDDS-8247):

      2023-03-22 16:00:35 ************************************************************/
      2023-03-22 16:00:35 2023-03-22 23:00:35,102 [main] INFO recon.ReconServer: registered UNIX signal handlers for [TERM, HUP, INT]
      2023-03-22 16:00:35 Exception in thread "main" java.lang.ExceptionInInitializerError
      2023-03-22 16:00:35     at com.google.inject.internal.cglib.reflect.$FastClassEmitter.<init>(FastClassEmitter.java:67)
      2023-03-22 16:00:35     at com.google.inject.internal.cglib.reflect.$FastClass$Generator.generateClass(FastClass.java:72)
      2023-03-22 16:00:35     at com.google.inject.internal.cglib.core.$DefaultGeneratorStrategy.generate(DefaultGeneratorStrategy.java:25)
      2023-03-22 16:00:35     at com.google.inject.internal.cglib.core.$AbstractClassGenerator.create(AbstractClassGenerator.java:216)
      2023-03-22 16:00:35     at com.google.inject.internal.cglib.reflect.$FastClass$Generator.create(FastClass.java:64)
      2023-03-22 16:00:35     at com.google.inject.internal.BytecodeGen.newFastClass(BytecodeGen.java:204)
      2023-03-22 16:00:35     at com.google.inject.internal.ProviderMethod$FastClassProviderMethod.<init>(ProviderMethod.java:256)
      2023-03-22 16:00:35     at com.google.inject.internal.ProviderMethod.create(ProviderMethod.java:71)
      2023-03-22 16:00:35     at com.google.inject.internal.ProviderMethodsModule.createProviderMethod(ProviderMethodsModule.java:275)
      2023-03-22 16:00:35     at com.google.inject.internal.ProviderMethodsModule.getProviderMethods(ProviderMethodsModule.java:144)
      2023-03-22 16:00:35     at com.google.inject.internal.ProviderMethodsModule.configure(ProviderMethodsModule.java:123)
      2023-03-22 16:00:35     at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
      2023-03-22 16:00:35     at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:349)
      2023-03-22 16:00:35     at com.google.inject.AbstractModule.install(AbstractModule.java:122)
      2023-03-22 16:00:35     at org.apache.hadoop.ozone.recon.ReconControllerModule.configure(ReconControllerModule.java:106)
      2023-03-22 16:00:35     at com.google.inject.AbstractModule.configure(AbstractModule.java:62)
      2023-03-22 16:00:35     at com.google.inject.spi.Elements$RecordingBinder.install(Elements.java:340)
      2023-03-22 16:00:35     at com.google.inject.spi.Elements.getElements(Elements.java:110)
      2023-03-22 16:00:35     at com.google.inject.internal.InjectorShell$Builder.build(InjectorShell.java:138)
      2023-03-22 16:00:35     at com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:104)
      2023-03-22 16:00:35     at com.google.inject.Guice.createInjector(Guice.java:96)
      2023-03-22 16:00:35     at com.google.inject.Guice.createInjector(Guice.java:73)
      2023-03-22 16:00:35     at com.google.inject.Guice.createInjector(Guice.java:62)
      2023-03-22 16:00:35     at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:100)
      2023-03-22 16:00:35     at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:65)
      2023-03-22 16:00:35     at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
      2023-03-22 16:00:35     at picocli.CommandLine.access$1300(CommandLine.java:145)
      2023-03-22 16:00:35     at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
      2023-03-22 16:00:35     at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
      2023-03-22 16:00:35     at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
      2023-03-22 16:00:35     at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
      2023-03-22 16:00:35     at picocli.CommandLine.execute(CommandLine.java:2078)
      2023-03-22 16:00:35     at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:100)
      2023-03-22 16:00:35     at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:91)
      2023-03-22 16:00:35     at org.apache.hadoop.ozone.recon.ReconServer.main(ReconServer.java:87)
      2023-03-22 16:00:35 Caused by: java.lang.reflect.InaccessibleObjectException: Unable to make protected final java.lang.Class java.lang.ClassLoader.defineClass(java.lang.String,byte[],int,int,java.security.ProtectionDomain) throws java.lang.ClassFormatError accessible: module java.base does not "opens java.lang" to unnamed module @6e20b53a
      2023-03-22 16:00:35     at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:354)
      2023-03-22 16:00:35     at java.base/java.lang.reflect.AccessibleObject.checkCanSetAccessible(AccessibleObject.java:297)
      2023-03-22 16:00:35     at java.base/java.lang.reflect.Method.checkCanSetAccessible(Method.java:199)
      2023-03-22 16:00:35     at java.base/java.lang.reflect.Method.setAccessible(Method.java:193)
      2023-03-22 16:00:35     at com.google.inject.internal.cglib.core.$ReflectUtils$2.run(ReflectUtils.java:56)
      2023-03-22 16:00:35     at java.base/java.security.AccessController.doPrivileged(AccessController.java:318)
      2023-03-22 16:00:35     at com.google.inject.internal.cglib.core.$ReflectUtils.<clinit>(ReflectUtils.java:46)
      2023-03-22 16:00:35     ... 35 more
      2023-03-22 16:00:35 2023-03-22 23:00:35,365 [shutdown-hook-0] INFO recon.ReconServer: SHUTDOWN_MSG: 
      2023-03-22 16:00:35 /************************************************************
      2023-03-22 16:00:35 SHUTDOWN_MSG: Shutting down ReconServer at db96dd3198fc/172.22.0.4
      2023-03-22 16:00:35 ************************************************************/
      

      Might have something to do with the dependency injection library?

      https://github.com/google/guice/issues/1536#issuecomment-963621665

      Try bumping up Guice (HDDS-8263).

      Update 1:

      With HDDS-8263 done and guice bumped, Recon is crashing with a different exception on startup (still using the same dev docker with JDK 17):

      2023-03-25 01:05:35,892 [Listener at 0.0.0.0/9891] INFO recon.ReconServer: Recon server initialized successfully!
      2023-03-25 01:05:35,892 [Listener at 0.0.0.0/9891] INFO recon.ReconServer: Starting Recon server
      2023-03-25 01:05:35,928 [Listener at 0.0.0.0/9891] INFO impl.MetricsConfig: Loaded properties from hadoop-metrics2.properties
      2023-03-25 01:05:35,934 [Listener at 0.0.0.0/9891] INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
      2023-03-25 01:05:35,934 [Listener at 0.0.0.0/9891] INFO impl.MetricsSystemImpl: Recon metrics system started
      2023-03-25 01:05:36,056 [Listener at 0.0.0.0/9891] INFO http.HttpServer2: Jetty bound to port 9888
      2023-03-25 01:05:36,057 [Listener at 0.0.0.0/9891] INFO server.Server: jetty-9.4.49.v20220914; built: 2022-09-14T01:07:36.601Z; git: 4231a3b2e4cb8548a412a789936d640a97b1aa0a; jvm 17.0.6+10-LTS
      2023-03-25 01:05:36,071 [Listener at 0.0.0.0/9891] INFO server.session: DefaultSessionIdManager workerName=node0
      2023-03-25 01:05:36,071 [Listener at 0.0.0.0/9891] INFO server.session: No SessionScavenger set, using defaults
      2023-03-25 01:05:36,072 [Listener at 0.0.0.0/9891] INFO server.session: node0 Scavenging every 660000ms
      2023-03-25 01:05:36,080 [Listener at 0.0.0.0/9891] INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@37e7c4cc{logs,/logs,file:///var/log/hadoop/,AVAILABLE}
      2023-03-25 01:05:36,080 [Listener at 0.0.0.0/9891] INFO handler.ContextHandler: Started o.e.j.s.ServletContextHandler@6aea99e7{static,/static,jar:file:/opt/hadoop/share/ozone/lib/ozone-recon-1.4.0-SNAPSHOT.jar!/webapps/static,AVAILABLE}
      Mar 25, 2023 1:05:37 AM com.sun.xml.bind.v2.runtime.reflect.opt.Injector <clinit>
      SEVERE: null
      java.security.PrivilegedActionException: java.lang.NoSuchMethodException: sun.misc.Unsafe.defineClass(java.lang.String,[B,int,int,java.lang.ClassLoader,java.security.ProtectionDomain)
              at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)
              at com.sun.xml.bind.v2.runtime.reflect.opt.Injector.<clinit>(Injector.java:197)
              at com.sun.xml.bind.v2.runtime.reflect.opt.AccessorInjector.prepare(AccessorInjector.java:81)
              at com.sun.xml.bind.v2.runtime.reflect.opt.OptimizedAccessorFactory.get(OptimizedAccessorFactory.java:179)
              at com.sun.xml.bind.v2.runtime.reflect.Accessor$FieldReflection.optimize(Accessor.java:285)
              at com.sun.xml.bind.v2.runtime.property.ArrayProperty.<init>(ArrayProperty.java:68)
              at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.<init>(ArrayERProperty.java:88)
              at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.<init>(ArrayElementProperty.java:100)
              at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.<init>(ArrayElementNodeProperty.java:62)
              at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
              at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
              at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
              at com.sun.xml.bind.v2.runtime.property.PropertyFactory.create(PropertyFactory.java:128)
              at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.<init>(ClassBeanInfoImpl.java:181)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl.getOrCreate(JAXBContextImpl.java:514)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:331)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:139)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1156)
              at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:165)
              at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:289)
              at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
              at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.base/java.lang.reflect.Method.invoke(Method.java:568)
              at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:217)
              at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:175)
              at javax.xml.bind.ContextFinder.find(ContextFinder.java:353)
              at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:508)
              at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:465)
              at org.glassfish.jersey.server.wadl.internal.WadlApplicationContextImpl.getJAXBContextFromWadlGenerator(WadlApplicationContextImpl.java:121)
              at org.glassfish.jersey.server.wadl.internal.WadlApplicationContextImpl.isJaxbImplAvailable(WadlApplicationContextImpl.java:270)
              at org.glassfish.jersey.server.wadl.WadlFeature.configure(WadlFeature.java:65)
              at org.glassfish.jersey.model.internal.CommonConfig.configureFeatures(CommonConfig.java:728)
              at org.glassfish.jersey.model.internal.CommonConfig.configureMetaProviders(CommonConfig.java:647)
              at org.glassfish.jersey.server.ResourceConfig.configureMetaProviders(ResourceConfig.java:823)
              at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:328)
              at org.glassfish.jersey.server.ApplicationHandler.lambda$initialize$1(ApplicationHandler.java:293)
              at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
              at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
              at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:232)
              at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:292)
              at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:259)
              at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:311)
              at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:154)
              at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:347)
              at javax.servlet.GenericServlet.init(GenericServlet.java:244)
              at com.google.inject.servlet.ServletDefinition.init(ServletDefinition.java:119)
              at com.google.inject.servlet.ManagedServletPipeline.init(ManagedServletPipeline.java:80)
              at com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:102)
              at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:232)
              at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:140)
              at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
              at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
              at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
              at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
              at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:774)
              at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
              at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
              at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
              at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
              at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
              at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
              at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
              at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
              at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
              at org.eclipse.jetty.server.Server.start(Server.java:423)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
              at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
              at org.eclipse.jetty.server.Server.doStart(Server.java:387)
              at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
              at org.apache.hadoop.hdds.server.http.HttpServer2.start(HttpServer2.java:1198)
              at org.apache.hadoop.hdds.server.http.BaseHttpServer.start(BaseHttpServer.java:312)
              at org.apache.hadoop.ozone.recon.ReconServer.start(ReconServer.java:238)
              at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:151)
              at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:65)
              at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
              at picocli.CommandLine.access$1300(CommandLine.java:145)
              at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
              at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
              at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
              at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
              at picocli.CommandLine.execute(CommandLine.java:2078)
              at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:100)
              at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:91)
              at org.apache.hadoop.ozone.recon.ReconServer.main(ReconServer.java:87)
      Caused by: java.lang.NoSuchMethodException: sun.misc.Unsafe.defineClass(java.lang.String,[B,int,int,java.lang.ClassLoader,java.security.ProtectionDomain)
              at java.base/java.lang.Class.getMethod(Class.java:2227)
              at com.sun.xml.bind.v2.runtime.reflect.opt.Injector$3.run(Injector.java:201)
              at com.sun.xml.bind.v2.runtime.reflect.opt.Injector$3.run(Injector.java:197)
              at java.base/java.security.AccessController.doPrivileged(AccessController.java:569)
              ... 89 more
      2023-03-25 01:05:37,126 [Listener at 0.0.0.0/9891] WARN webapp.WebAppContext: Failed startup of context o.e.j.w.WebAppContext@76bdfc19{recon,/,file:///data/metadata/webserver/jetty-0_0_0_0-9888-ozone-recon-1_4_0-SNAPSHOT_jar-_-any-4970389891519408937/webapp/,UNAVAILABLE}{jar:file:/opt/hadoop/share/ozone/lib/ozone-recon-1.4.0-SNAPSHOT.jar!/webapps/recon}
      java.lang.NullPointerException: Cannot invoke "java.lang.reflect.Method.invoke(Object, Object[])" because "com.sun.xml.bind.v2.runtime.reflect.opt.Injector.defineClass" is null
              at com.sun.xml.bind.v2.runtime.reflect.opt.Injector.inject(Injector.java:311)
              at com.sun.xml.bind.v2.runtime.reflect.opt.Injector.inject(Injector.java:97)
              at com.sun.xml.bind.v2.runtime.reflect.opt.AccessorInjector.prepare(AccessorInjector.java:87)
              at com.sun.xml.bind.v2.runtime.reflect.opt.OptimizedAccessorFactory.get(OptimizedAccessorFactory.java:179)
              at com.sun.xml.bind.v2.runtime.reflect.Accessor$FieldReflection.optimize(Accessor.java:285)
              at com.sun.xml.bind.v2.runtime.property.ArrayProperty.<init>(ArrayProperty.java:68)
              at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.<init>(ArrayERProperty.java:88)
              at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.<init>(ArrayElementProperty.java:100)
              at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.<init>(ArrayElementNodeProperty.java:62)
              at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
              at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
              at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
              at com.sun.xml.bind.v2.runtime.property.PropertyFactory.create(PropertyFactory.java:128)
              at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.<init>(ClassBeanInfoImpl.java:181)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl.getOrCreate(JAXBContextImpl.java:514)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:331)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:139)
              at com.sun.xml.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1156)
              at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:165)
              at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:289)
              at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
              at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
              at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
              at java.base/java.lang.reflect.Method.invoke(Method.java:568)
              at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:217)
              at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:175)
              at javax.xml.bind.ContextFinder.find(ContextFinder.java:353)
              at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:508)
              at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:465)
              at org.glassfish.jersey.server.wadl.internal.WadlApplicationContextImpl.getJAXBContextFromWadlGenerator(WadlApplicationContextImpl.java:121)
              at org.glassfish.jersey.server.wadl.internal.WadlApplicationContextImpl.isJaxbImplAvailable(WadlApplicationContextImpl.java:270)
              at org.glassfish.jersey.server.wadl.WadlFeature.configure(WadlFeature.java:65)
              at org.glassfish.jersey.model.internal.CommonConfig.configureFeatures(CommonConfig.java:728)
              at org.glassfish.jersey.model.internal.CommonConfig.configureMetaProviders(CommonConfig.java:647)
              at org.glassfish.jersey.server.ResourceConfig.configureMetaProviders(ResourceConfig.java:823)
              at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:328)
              at org.glassfish.jersey.server.ApplicationHandler.lambda$initialize$1(ApplicationHandler.java:293)
              at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
              at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
              at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:232)
              at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:292)
              at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:259)
              at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:311)
              at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:154)
              at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:347)
              at javax.servlet.GenericServlet.init(GenericServlet.java:244)
              at com.google.inject.servlet.ServletDefinition.init(ServletDefinition.java:119)
              at com.google.inject.servlet.ManagedServletPipeline.init(ManagedServletPipeline.java:80)
              at com.google.inject.servlet.ManagedFilterPipeline.initPipeline(ManagedFilterPipeline.java:102)
              at com.google.inject.servlet.GuiceFilter.init(GuiceFilter.java:232)
              at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:140)
              at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
              at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
              at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:734)
              at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
              at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:774)
              at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
              at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
              at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
              at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
              at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
              at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
              at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
              at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
              at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
              at org.eclipse.jetty.server.Server.start(Server.java:423)
              at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
              at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
              at org.eclipse.jetty.server.Server.doStart(Server.java:387)
              at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
              at org.apache.hadoop.hdds.server.http.HttpServer2.start(HttpServer2.java:1198)
              at org.apache.hadoop.hdds.server.http.BaseHttpServer.start(BaseHttpServer.java:312)
              at org.apache.hadoop.ozone.recon.ReconServer.start(ReconServer.java:238)
              at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:151)
              at org.apache.hadoop.ozone.recon.ReconServer.call(ReconServer.java:65)
              at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
              at picocli.CommandLine.access$1300(CommandLine.java:145)
              at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
              at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
              at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
              at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
              at picocli.CommandLine.execute(CommandLine.java:2078)
              at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:100)
              at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:91)
              at org.apache.hadoop.ozone.recon.ReconServer.main(ReconServer.java:87)
      

      This looks to be same source of problem as S3g: HDDS-8249. Might need to bump jetty to at least 10.x (Tried 9.4.51.v20230217, same issue). However, jetty 10.x/11.x requires the use of JDK 11 to compile.

      Attachments

        Issue Links

          Activity

            People

              Unassigned Unassigned
              smeng Siyao Meng
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: