Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-20602

Getting exception Connecting kinesis with mfa

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Incomplete
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: Connectors / Kinesis
    • Labels:
      None

      Description

      I am trying to connect to kinesis (running with mfa) with the following configuration :

       

      val producerConfig = new Properties()
      producerConfig.put(AWSConfigConstants.AWS_REGION, awsRegion)
      producerConfig.put(AWSConfigConstants.AWS_ACCESS_KEY_ID, awsAccessKey)
      producerConfig.put(AWSConfigConstants.AWS_SECRET_ACCESS_KEY, awsSecretAccessKey)
      producerConfig.put(com.amazonaws.auth.profile.internal.ProfileKeyConstants.AWS_SESSION_TOKEN, awsSessionToken)
      

      with a very simple pipeline :

       

      val producer = new FlinkKinesisProducer(new SimpleStringSchema(), producerConfig)
      producer.setFailOnError(true)
      producer.setDefaultStream(outputStreamName)
      producer.setDefaultPartition("0")
      env.fromElements("a", "b", "c").addSink(producer)
      env.execute()
      

      the results with:

      15:30:44,292 WARN org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.LogInputStreamReader - [2020-12-14 15:30:44.292188] [0x0000cb5f][0x000070000512c000] [warning] [AWS Log: WARN](AWSClient)If the signature check failed. This could be because of a time skew. Attempting to adjust the signer.
      15:30:44,378 INFO org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.LogInputStreamReader - [2020-12-14 15:30:44.377865] [0x0000cb5b][0x00007000082c1000] [info] [shard_map.cc:87] Updating shard map for stream "ExampleOutputStream"
      15:30:44,396 WARN org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.LogInputStreamReader - [2020-12-14 15:30:44.396208] [0x0000cb55][0x0000700002a3e000] [warning] [AWS Log: WARN](AWSErrorMarshaller)Encountered AWSError 'UnrecognizedClientException': The security token included in the request is invalid.
      15:30:44,396 ERROR org.apache.flink.kinesis.shaded.com.amazonaws.services.kinesis.producer.LogInputStreamReader - [2020-12-14 15:30:44.396256] [0x0000cb55][0x0000700002a3e000] [error] [AWS Log: ERROR](AWSClient)HTTP response code: 400
      Exception name: UnrecognizedClientException
      Error message: The security token included in the request is invalid.
      6 response headers:
      connection : close

      I double check that all keys are correct using the same keys that works perfectly when i execute commands from the cli. also when removing the mfa from kinesis the pipeline works as expected

        Attachments

        Issue Links

          Activity

            People

            • Assignee:
              dannycranmer Danny Cranmer
              Reporter:
              avilevi Avi Levi

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment