Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-8246 Java 11/17 Support and Testing
  3. HDDS-8249

[JDK17] S3 Gateway crashes on startup with JDK 17: Remove sun.misc.Unsafe or work around it

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Resolved
    • Major
    • Resolution: Works for Me
    • None
    • None
    • S3
    • None

    Description

      S3g crashed on startup in Docker dev environment with JDK 17 (follow HDDS-8247 for the docker image for repro):

      2023-03-22 15:52:56 SEVERE: null
2023-03-22 15:52:56 java.security.PrivilegedActionException: java.lang.NoSuchMethodException: sun.misc.Unsafe.defineClass(java.lang.String,[B,int,int,java.lang.ClassLoader,java.security.ProtectionDomain)
2023-03-22 15:52:56     at java.base/java.security.AccessController.doPrivileged(AccessController.java:573)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.reflect.opt.Injector.<clinit>(Injector.java:197)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.reflect.opt.AccessorInjector.prepare(AccessorInjector.java:81)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.reflect.opt.OptimizedAccessorFactory.get(OptimizedAccessorFactory.java:179)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.reflect.Accessor$FieldReflection.optimize(Accessor.java:285)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.property.ArrayProperty.<init>(ArrayProperty.java:68)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.property.ArrayERProperty.<init>(ArrayERProperty.java:88)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.property.ArrayElementProperty.<init>(ArrayElementProperty.java:100)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.property.ArrayElementNodeProperty.<init>(ArrayElementNodeProperty.java:62)
2023-03-22 15:52:56     at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
2023-03-22 15:52:56     at java.base/jdk.internal.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:77)
2023-03-22 15:52:56     at java.base/jdk.internal.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
2023-03-22 15:52:56     at java.base/java.lang.reflect.Constructor.newInstanceWithCaller(Constructor.java:499)
2023-03-22 15:52:56     at java.base/java.lang.reflect.Constructor.newInstance(Constructor.java:480)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.property.PropertyFactory.create(PropertyFactory.java:128)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.ClassBeanInfoImpl.<init>(ClassBeanInfoImpl.java:181)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.JAXBContextImpl.getOrCreate(JAXBContextImpl.java:514)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:331)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.JAXBContextImpl.<init>(JAXBContextImpl.java:139)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.runtime.JAXBContextImpl$JAXBContextBuilder.build(JAXBContextImpl.java:1156)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:165)
2023-03-22 15:52:56     at com.sun.xml.bind.v2.ContextFactory.createContext(ContextFactory.java:289)
2023-03-22 15:52:56     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
2023-03-22 15:52:56     at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
2023-03-22 15:52:56     at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
2023-03-22 15:52:56     at java.base/java.lang.reflect.Method.invoke(Method.java:568)
2023-03-22 15:52:56     at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:217)
2023-03-22 15:52:56     at javax.xml.bind.ContextFinder.newInstance(ContextFinder.java:175)
2023-03-22 15:52:56     at javax.xml.bind.ContextFinder.find(ContextFinder.java:353)
2023-03-22 15:52:56     at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:508)
2023-03-22 15:52:56     at javax.xml.bind.JAXBContext.newInstance(JAXBContext.java:465)
2023-03-22 15:52:56     at org.glassfish.jersey.server.wadl.internal.WadlApplicationContextImpl.getJAXBContextFromWadlGenerator(WadlApplicationContextImpl.java:121)
2023-03-22 15:52:56     at org.glassfish.jersey.server.wadl.internal.WadlApplicationContextImpl.isJaxbImplAvailable(WadlApplicationContextImpl.java:270)
2023-03-22 15:52:56     at org.glassfish.jersey.server.wadl.WadlFeature.configure(WadlFeature.java:65)
2023-03-22 15:52:56     at org.glassfish.jersey.model.internal.CommonConfig.configureFeatures(CommonConfig.java:728)
2023-03-22 15:52:56     at org.glassfish.jersey.model.internal.CommonConfig.configureMetaProviders(CommonConfig.java:647)
2023-03-22 15:52:56     at org.glassfish.jersey.server.ResourceConfig.configureMetaProviders(ResourceConfig.java:823)
2023-03-22 15:52:56     at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:328)
2023-03-22 15:52:56     at org.glassfish.jersey.server.ApplicationHandler.lambda$initialize$1(ApplicationHandler.java:293)
2023-03-22 15:52:56     at org.glassfish.jersey.internal.Errors.process(Errors.java:292)
2023-03-22 15:52:56     at org.glassfish.jersey.internal.Errors.process(Errors.java:274)
2023-03-22 15:52:56     at org.glassfish.jersey.internal.Errors.processWithException(Errors.java:232)
2023-03-22 15:52:56     at org.glassfish.jersey.server.ApplicationHandler.initialize(ApplicationHandler.java:292)
2023-03-22 15:52:56     at org.glassfish.jersey.server.ApplicationHandler.<init>(ApplicationHandler.java:259)
2023-03-22 15:52:56     at org.glassfish.jersey.servlet.WebComponent.<init>(WebComponent.java:311)
2023-03-22 15:52:56     at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:154)
2023-03-22 15:52:56     at org.glassfish.jersey.servlet.ServletContainer.init(ServletContainer.java:347)
2023-03-22 15:52:56     at javax.servlet.GenericServlet.init(GenericServlet.java:244)
2023-03-22 15:52:56     at org.eclipse.jetty.servlet.ServletHolder$Wrapper.init(ServletHolder.java:1345)
2023-03-22 15:52:56     at org.eclipse.jetty.servlet.ServletHolder.initServlet(ServletHolder.java:632)
2023-03-22 15:52:56     at org.eclipse.jetty.servlet.ServletHolder.initialize(ServletHolder.java:415)
2023-03-22 15:52:56     at org.eclipse.jetty.servlet.ServletHandler.lambda$initialize$0(ServletHandler.java:750)
2023-03-22 15:52:56     at java.base/java.util.stream.SortedOps$SizedRefSortingSink.end(SortedOps.java:357)
2023-03-22 15:52:56     at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:510)
2023-03-22 15:52:56     at java.base/java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:499)
2023-03-22 15:52:56     at java.base/java.util.stream.StreamSpliterators$WrappingSpliterator.forEachRemaining(StreamSpliterators.java:310)
2023-03-22 15:52:56     at java.base/java.util.stream.Streams$ConcatSpliterator.forEachRemaining(Streams.java:735)
2023-03-22 15:52:56     at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:762)
2023-03-22 15:52:56     at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:774)
2023-03-22 15:52:56     at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
2023-03-22 15:52:56     at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
2023-03-22 15:52:56     at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
2023-03-22 15:52:56     at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
2023-03-22 15:52:56     at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
2023-03-22 15:52:56     at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
2023-03-22 15:52:56     at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
2023-03-22 15:52:56     at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
2023-03-22 15:52:56     at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
2023-03-22 15:52:56     at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
2023-03-22 15:52:56     at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
2023-03-22 15:52:56     at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
2023-03-22 15:52:56     at org.eclipse.jetty.server.Server.start(Server.java:423)
2023-03-22 15:52:56     at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
2023-03-22 15:52:56     at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
2023-03-22 15:52:56     at org.eclipse.jetty.server.Server.doStart(Server.java:387)
2023-03-22 15:52:56     at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
2023-03-22 15:52:56     at org.apache.hadoop.hdds.server.http.HttpServer2.start(HttpServer2.java:1198)
2023-03-22 15:52:56     at org.apache.hadoop.hdds.server.http.BaseHttpServer.start(BaseHttpServer.java:312)
2023-03-22 15:52:56     at org.apache.hadoop.ozone.s3.Gateway.start(Gateway.java:95)
2023-03-22 15:52:56     at org.apache.hadoop.ozone.s3.Gateway.call(Gateway.java:75)
2023-03-22 15:52:56     at org.apache.hadoop.ozone.s3.Gateway.call(Gateway.java:48)
2023-03-22 15:52:56     at picocli.CommandLine.executeUserObject(CommandLine.java:1953)
2023-03-22 15:52:56     at picocli.CommandLine.access$1300(CommandLine.java:145)
2023-03-22 15:52:56     at picocli.CommandLine$RunLast.executeUserObjectOfLastSubcommandWithSameParent(CommandLine.java:2352)
2023-03-22 15:52:56     at picocli.CommandLine$RunLast.handle(CommandLine.java:2346)
2023-03-22 15:52:56     at picocli.CommandLine$RunLast.handle(CommandLine.java:2311)
2023-03-22 15:52:56     at picocli.CommandLine$AbstractParseResultHandler.execute(CommandLine.java:2179)
2023-03-22 15:52:56     at picocli.CommandLine.execute(CommandLine.java:2078)
2023-03-22 15:52:56     at org.apache.hadoop.hdds.cli.GenericCli.execute(GenericCli.java:100)
2023-03-22 15:52:56     at org.apache.hadoop.hdds.cli.GenericCli.run(GenericCli.java:91)
2023-03-22 15:52:56     at org.apache.hadoop.ozone.s3.Gateway.main(Gateway.java:63)

      Immediately reproducible with ./compose/ozone compose yaml.

      Though it looks like it is coming from picocli. Trying bumping it up? Need to dig deeper.

      Attachments

        Issue Links

          is blocked by

          Sub-task - The sub-task of the issue HDDS-8280 [Discussion][JDK17] Bump jetty from 9.4.x to 11.x/12.x

          • Major - Major loss of function.
          • Open
          is fixed by

          Task - A task that needs to be done. HDDS-9829 Bump jaxb-api to 2.3.1

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              smeng Siyao Meng
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: