[FLINK-34490] flink-connector-kinesis not correctly supporting credential chaining

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: aws-connector-4.2.0, 1.17.2
• Component/s: Connectors / Kinesis
• Assignee: Aleksandr Pilipenko
• Reporter: Eddie Ramirez
• Created: 21/Feb/24 22:33
• Updated: 23/Feb/24 21:19

Description

When using AWS credential chaining, `flink-connector-kinesis` does not correctly follow the chain of credentials.

 

Expected Result

 `flink-connector-kinesis`  should follow the `source_profile` for each respective profile in `~/.aws/config` to ultimately determine credentials.

 

Observed Result

 `flink-connector-kinesis` only follows the first matching `source_profile` specified in `~/.aws/config` and then errors out because there is no credentials for that profile.

org.apache.flink.kinesis.shaded.com.amazonaws.SdkClientException: Unable to load credentials into profile [profile intermediate-role]: AWS Access Key ID is not specified

 

Configuration

connector config

aws.credentials.provider: PROFILE
aws.credentials.profile.name: flink-access-role

 

aws `~/.aws/config` file

[profile flink-access-role]
role_arn = arn:aws:iam::xxxxxxxxx:role/flink-access-role
source_profile = intermediate-role

[profile intermediate-role]
role_arn = arn:aws:iam::xxxxxxxxx:role/intermediate-role
source_profile = aws-sso-role

[profile aws-sso-role]
sso_session = idc
sso_role_name = xxxxx
sso_account_id = xxxxx
credential_process = aws configure export-credentials --profile=aws-sso-role

[sso-session idc]
sso_start_url = xxxxx
sso_region = xxxxx
sso_registration_scopes = sso:account:access