Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-16054

Skipped attempt to connect to other TcpDiscovery nodes if SSL is enabled and first node from list is down.

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Reopened
    • Major
    • Resolution: Unresolved
    • 2.11
    • None
    • None
    • Docs Required, Release Notes Required

    Description

      After IGNITE-4147, if node is attempting to connect to the first node and it is down, with enabled SSL, we mistakenly treat SSLException as handshake error and fail fast without attempts to connect to others.

      We should examine cause of SSLException anf if it is an instance of IOException, we should attempt to connect to other nodes.

      2021-12-02 15:19:06,003 [main] [ERROR] (org.apache.ignite.internal.IgniteKernal%XXXX7d657515-18b3-4189-a920-ba5040ad24ae) [org.apache.ignite.logger.java.JavaLogger::error:310] mdc:()| Got exception while starting (will rollback startup routine).
      org.apache.ignite.IgniteCheckedException: Failed to start manager: GridManagerAdapter [enabled=true, name=org.apache.ignite.internal.managers.discovery.GridDiscoveryManager]
      	at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1973)
      	at org.apache.ignite.internal.IgniteKernal.start(IgniteKernal.java:1324)
      	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start0(IgnitionEx.java:2112)
      	at org.apache.ignite.internal.IgnitionEx$IgniteNamedInstance.start(IgnitionEx.java:1758)
      	at org.apache.ignite.internal.IgnitionEx.start0(IgnitionEx.java:1143)
      	at org.apache.ignite.internal.IgnitionEx.startConfigurations(IgnitionEx.java:1061)
      	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:947)
      	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:846)
      	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:716)
      	at org.apache.ignite.internal.IgnitionEx.start(IgnitionEx.java:685)
      	at org.apache.ignite.Ignition.start(Ignition.java:353)
      	at ru.XX.XX.common.core.IgniteCacheConfiguration.igniteInstance(IgniteCacheConfiguration.java:20)
      	at ru.XX.XX.common.core.IgniteCacheConfiguration$$EnhancerBySpringCGLIB$$d8cb8bdc.CGLIB$igniteInstance$1(<generated>)
      	at ru.XX.XX.common.core.IgniteCacheConfiguration$$EnhancerBySpringCGLIB$$d8cb8bdc$$FastClassBySpringCGLIB$$38a3b050.invoke(<generated>)
      	at org.springframework.cglib.proxy.MethodProxy.invokeSuper(MethodProxy.java:244)
      	at org.springframework.context.annotation.ConfigurationClassEnhancer$BeanMethodInterceptor.intercept(ConfigurationClassEnhancer.java:331)
      	at ru.XX.XX.common.core.IgniteCacheConfiguration$$EnhancerBySpringCGLIB$$d8cb8bdc.igniteInstance(<generated>)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.springframework.beans.factory.support.SimpleInstantiationStrategy.instantiate(SimpleInstantiationStrategy.java:154)
      	at org.springframework.beans.factory.support.ConstructorResolver.instantiate(ConstructorResolver.java:653)
      	at org.springframework.beans.factory.support.ConstructorResolver.instantiateUsingFactoryMethod(ConstructorResolver.java:638)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.instantiateUsingFactoryMethod(AbstractAutowireCapableBeanFactory.java:1334)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1177)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:564)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
      	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
      	at org.springframework.beans.factory.config.DependencyDescriptor.resolveCandidate(DependencyDescriptor.java:276)
      	at org.springframework.beans.factory.support.DefaultListableBeanFactory.doResolveDependency(DefaultListableBeanFactory.java:1380)
      	at org.springframework.beans.factory.support.DefaultListableBeanFactory.resolveDependency(DefaultListableBeanFactory.java:1300)
      	at org.springframework.beans.factory.support.ConstructorResolver.resolveAutowiredArgument(ConstructorResolver.java:887)
      	at org.springframework.beans.factory.support.ConstructorResolver.createArgumentArray(ConstructorResolver.java:791)
      	at org.springframework.beans.factory.support.ConstructorResolver.autowireConstructor(ConstructorResolver.java:229)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.autowireConstructor(AbstractAutowireCapableBeanFactory.java:1354)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBeanInstance(AbstractAutowireCapableBeanFactory.java:1204)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.doCreateBean(AbstractAutowireCapableBeanFactory.java:564)
      	at org.springframework.beans.factory.support.AbstractAutowireCapableBeanFactory.createBean(AbstractAutowireCapableBeanFactory.java:524)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.lambda$doGetBean$0(AbstractBeanFactory.java:335)
      	at org.springframework.beans.factory.support.DefaultSingletonBeanRegistry.getSingleton(DefaultSingletonBeanRegistry.java:234)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.doGetBean(AbstractBeanFactory.java:333)
      	at org.springframework.beans.factory.support.AbstractBeanFactory.getBean(AbstractBeanFactory.java:208)
      	at org.springframework.beans.factory.support.DefaultListableBeanFactory.preInstantiateSingletons(DefaultListableBeanFactory.java:944)
      	at org.springframework.context.support.AbstractApplicationContext.finishBeanFactoryInitialization(AbstractApplicationContext.java:918)
      	at org.springframework.context.support.AbstractApplicationContext.refresh(AbstractApplicationContext.java:583)
      	at org.springframework.boot.web.servlet.context.ServletWebServerApplicationContext.refresh(ServletWebServerApplicationContext.java:145)
      	at org.springframework.boot.SpringApplication.refresh(SpringApplication.java:754)
      	at org.springframework.boot.SpringApplication.refreshContext(SpringApplication.java:434)
      	at org.springframework.boot.SpringApplication.run(SpringApplication.java:338)
      	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1343)
      	at org.springframework.boot.SpringApplication.run(SpringApplication.java:1332)
      	at ru.XX.XX.DigitalApiRunner.main(DigitalApiRunner.java:14)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.base/java.lang.reflect.Method.invoke(Method.java:566)
      	at org.springframework.boot.loader.MainMethodRunner.run(MainMethodRunner.java:49)
      	at org.springframework.boot.loader.Launcher.launch(Launcher.java:108)
      	at org.springframework.boot.loader.Launcher.launch(Launcher.java:58)
      	at org.springframework.boot.loader.JarLauncher.main(JarLauncher.java:88)
      Caused by: org.apache.ignite.IgniteCheckedException: Failed to start SPI: TcpDiscoverySpi [addrRslvr=null, sockTimeout=5000, ackTimeout=5000, marsh=JdkMarshaller [clsFilter=org.apache.ignite.marshaller.MarshallerUtils$1@787988f4], reconCnt=10, reconDelay=2000, maxAckTimeout=600000, soLinger=0, forceSrvMode=false, clientReconnectDisabled=false, internalLsnr=null, skipAddrsRandomization=false]
      	at org.apache.ignite.internal.managers.GridManagerAdapter.startSpi(GridManagerAdapter.java:281)
      	at org.apache.ignite.internal.managers.discovery.GridDiscoveryManager.start(GridDiscoveryManager.java:981)
      	at org.apache.ignite.internal.IgniteKernal.startManager(IgniteKernal.java:1968)
      	... 63 common frames omitted
      Caused by: org.apache.ignite.spi.IgniteSpiException: Unable to establish secure connection. Was remote cluster configured with SSL? [rmtAddr=XXX/10.XX.XX.XX:47500, errMsg="Connection reset"]
      	at org.apache.ignite.spi.discovery.tcp.ClientImpl.sendJoinRequest(ClientImpl.java:825)
      	at org.apache.ignite.spi.discovery.tcp.ClientImpl.joinTopology(ClientImpl.java:629)
      	at org.apache.ignite.spi.discovery.tcp.ClientImpl.access$1000(ClientImpl.java:150)
      	at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.tryJoin(ClientImpl.java:2108)
      	at org.apache.ignite.spi.discovery.tcp.ClientImpl$MessageWorker.body(ClientImpl.java:1751)
      	at org.apache.ignite.internal.util.worker.GridWorker.run(GridWorker.java:120)
      	at org.apache.ignite.spi.discovery.tcp.ClientImpl$1.body(ClientImpl.java:317)
      	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:58)
      Caused by: javax.net.ssl.SSLException: Connection reset
      	at java.base/sun.security.ssl.Alert.createSSLException(Alert.java:127)
      	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:320)
      	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:263)
      	at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:258)
      	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:137)
      	at java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1151)
      	at java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1062)
      	at java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:402)
      	at java.base/sun.security.ssl.SSLSocketImpl.ensureNegotiated(SSLSocketImpl.java:716)
      	at java.base/sun.security.ssl.SSLSocketImpl$AppOutputStream.write(SSLSocketImpl.java:969)
      	at java.base/java.io.OutputStream.write(OutputStream.java:122)
      	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.writeToSocket(TcpDiscoverySpi.java:1654)
      	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1592)
      	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.openSocket(TcpDiscoverySpi.java:1552)
      	at org.apache.ignite.spi.discovery.tcp.ClientImpl.sendJoinRequest(ClientImpl.java:734)
      	... 7 common frames omitted
      	Suppressed: java.net.SocketException: Broken pipe (Write failed)
      		at java.base/java.net.SocketOutputStream.socketWrite0(Native Method)
      		at java.base/java.net.SocketOutputStream.socketWrite(SocketOutputStream.java:110)
      		at java.base/java.net.SocketOutputStream.write(SocketOutputStream.java:150)
      		at java.base/sun.security.ssl.SSLSocketOutputRecord.encodeAlert(SSLSocketOutputRecord.java:81)
      		at java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:351)
      		... 20 common frames omitted
      Caused by: java.net.SocketException: Connection reset
      	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:186)
      	at java.base/java.net.SocketInputStream.read(SocketInputStream.java:140)
      	at java.base/sun.security.ssl.SSLSocketInputRecord.read(SSLSocketInputRecord.java:448)
      	at java.base/sun.security.ssl.SSLSocketInputRecord.decode(SSLSocketInputRecord.java:165)
      	at java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:108)
      	... 17 common frames omitted
      

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            shishkovilja Ilya Shishkov
            ivandasch Ivan Daschinsky

            Dates

              Created:
              Updated:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 0.5h
                0.5h

                Slack

                  Issue deployment