Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-19716

[Kinesis][EFO] Unable to use Assume Role with EFO record publisher

    XMLWordPrintableJSON

    Details

      Description

      Reproduction Steps

      • Setup an application to consume from a Kinesis Stream
      • Use ASSUME_ROLE credential provider 
      
      consumerConfig.setProperty(AWSConfigConstants.AWS_CREDENTIALS_PROVIDER, ASSUME_ROLE.name());
      consumerConfig.setProperty(AWSConfigConstants.AWS_ROLE_ARN, "<role-arn>");
      consumerConfig.setProperty(AWSConfigConstants.AWS_ROLE_SESSION_NAME, "test-efo");
      
      

       

      Expected Result

      • Consumer is able to authorise and consume from the stream

      Actual Result

      • The following error is thrown (full stack attached)
        • Caused by: org.apache.flink.kinesis.shaded.software.amazon.awssdk.core.exception.SdkClientException: Unable to load an HTTP implementation from any provider in the chain. You must declare a dependency on an appropriate HTTP implementation or pass in an SdkHttpClient explicitly to the client builder.

      Diagnosis

      This issue occurs because Assume Role credential provider requires a Sync HTTP Client. The Apache HTTP Client is on the classpath but it is not detected due to the shading relocation. It is looking for:

      • org.apache.flink.kinesis.shaded.software.amazon.awssdk.http.SdkHttpService

      However the service manifest is defining:

      • software.amazon.awssdk.http.SdkHttpService

      Solution

      • Add a service manifest such that the shaded HTTP client is used
      • Also needed to update the HTTP client/core version due to incompatibilities 

      Testing

      Tested using EFO and POLLING record consumer 

       

        Attachments

        1. FLINK-19716-error.txt
          9 kB
          Danny Cranmer

          Issue Links

            Activity

              People

              • Assignee:
                danny.cranmer Danny Cranmer
                Reporter:
                danny.cranmer Danny Cranmer
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: