Uploaded image for project: 'Ignite'
  1. Ignite
  2. IGNITE-9826

Ignite node with TcpDiscoveryS3IpFinder can hang while stopping

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.6
    • 2.8
    • aws
    • None

    Description

      Here is a stack trace:

      [15:46:11,081][SEVERE][tcp-disco-ip-finder-cleaner-#4%node1%][TcpDiscoverySpi] Failed to clean IP finder up.
      class org.apache.ignite.spi.IgniteSpiException: Failed to list objects in the bucket: rguseinov-ipfinder
      	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:200)
      	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.S3IpFinderReproducer$CustomS3IpFinder.getRegisteredAddresses(S3IpFinderReproducer.java:103)
      	at org.apache.ignite.spi.discovery.tcp.TcpDiscoverySpi.registeredAddresses(TcpDiscoverySpi.java:1900)
      	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.cleanIpFinder(ServerImpl.java:1998)
      	at org.apache.ignite.spi.discovery.tcp.ServerImpl$IpFinderCleaner.body(ServerImpl.java:1973)
      	at org.apache.ignite.spi.IgniteSpiThread.run(IgniteSpiThread.java:62)
      Caused by: com.amazonaws.SdkClientException: Failed to sanitize XML document destined for handler class com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser$ListBucketHandler
      	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:214)
      	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.parseListBucketObjectsResponse(XmlResponsesSaxParser.java:298)
      	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:70)
      	at com.amazonaws.services.s3.model.transform.Unmarshallers$ListObjectsUnmarshaller.unmarshall(Unmarshallers.java:59)
      	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:62)
      	at com.amazonaws.services.s3.internal.S3XmlResponseHandler.handle(S3XmlResponseHandler.java:31)
      	at com.amazonaws.http.response.AwsResponseHandlerAdapter.handle(AwsResponseHandlerAdapter.java:70)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.handleResponse(AmazonHttpClient.java:1501)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeOneRequest(AmazonHttpClient.java:1222)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeHelper(AmazonHttpClient.java:1035)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.doExecute(AmazonHttpClient.java:747)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.executeWithTimer(AmazonHttpClient.java:721)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.execute(AmazonHttpClient.java:704)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutor.access$500(AmazonHttpClient.java:672)
      	at com.amazonaws.http.AmazonHttpClient$RequestExecutionBuilderImpl.execute(AmazonHttpClient.java:654)
      	at com.amazonaws.http.AmazonHttpClient.execute(AmazonHttpClient.java:518)
      	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4137)
      	at com.amazonaws.services.s3.AmazonS3Client.invoke(AmazonS3Client.java:4079)
      	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:819)
      	at com.amazonaws.services.s3.AmazonS3Client.listObjects(AmazonS3Client.java:791)
      	at org.apache.ignite.spi.discovery.tcp.ipfinder.s3.TcpDiscoveryS3IpFinder.getRegisteredAddresses(TcpDiscoveryS3IpFinder.java:148)
      	... 5 more
      Caused by: com.amazonaws.AbortedException: 
      	at com.amazonaws.internal.SdkFilterInputStream.abortIfNeeded(SdkFilterInputStream.java:51)
      	at com.amazonaws.internal.SdkFilterInputStream.read(SdkFilterInputStream.java:71)
      	at com.amazonaws.event.ProgressInputStream.read(ProgressInputStream.java:181)
      	at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
      	at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
      	at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
      	at java.io.InputStreamReader.read(InputStreamReader.java:184)
      	at java.io.BufferedReader.read1(BufferedReader.java:210)
      	at java.io.BufferedReader.read(BufferedReader.java:286)
      	at java.io.Reader.read(Reader.java:140)
      	at com.amazonaws.services.s3.model.transform.XmlResponsesSaxParser.sanitizeXmlDocument(XmlResponsesSaxParser.java:186)
      	... 25 more
      

      Reproducer is attached S3IpFinderReproducer.java .

      Attachments

        1. S3IpFinderReproducer.java
          3 kB
          Roman Guseinov

        Issue Links

          Activity

            People

              guseinov Roman Guseinov
              guseinov Roman Guseinov
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: