Details
-
New Feature
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
Azure Kubernetes Service (AKS)
Description
We are trying to use jclouds by deploying S3Proxy in a Kubernetes cluster on AKS in order to translate AWS S3-compatible requests performed by our own software to Azure Blob Storage requests. Even though using an Azure storage account token and name as credentials works, we are having trouble authenticating using Azure Pod Identity credentials.
In a nutshell, Pod Identities allow pods running in an AKS cluster to retrieve a JWT from the Azure Instance Metadata Service and use it to assume a managed identity when accessing Azure services. We want to use this feature because it allows assigning fine-grained permissions to services running in the AKS cluster, and is considered best practice for applications running on AKS. For more information see:
https://docs.microsoft.com/en-us/azure/aks/use-azure-ad-pod-identity
https://docs.microsoft.com/en-us/azure/aks/operator-best-practices-identity#use-pod-managed-identities
When trying to use this feature in combination with jclouds and S3Proxy, we bumped across the following two problems:
1. As far as we can tell we cannot make jclouds automatically retrieve and refresh Pod Identity credentials from the instance metadata service.
We have attempted to use the azurecompute-arm compute service to retrieve instance metadata from within an AKS pod without providing credentials. However, in order to list nodes it tries to contact login.microsoft.com to create an OAuth2 token and fails with the following error:
Illegal character in path at index 29: https://login.microsoft.com/${azurecompute-arm.tenantId}/oauth2/token
Ideally, when jclouds is running in an AKS node, if credentials have not been provided it should attempt to automatically retrieve and refresh them. Note that automatically refreshing the credentials is important since Pod Identity credentials are short-lived.
2. jclouds does not seem to support authentication via Pod Identity tokens. Our understanding is that currently only Shared Key Authorization and Shared Access Signatures is supported:
https://docs.microsoft.com/en-us/rest/api/storageservices/authorize-with-shared-key
https://docs.microsoft.com/en-us/azure/storage/common/storage-sas-overview
These two authentication methods use the provided base64-encoded credentials to sign the request. Pod Identity tokens on the other hand are JWTs that should be included as-is in the request as a Bearer token. As a result, even if we manually retrieve the Pod Identity credentials and pass them to jclouds, requests fail with the following error due to trying to base64-decode them:
com.google.common.io.BaseEncoding$DecodingException: Unrecognized character: .
Is there a way to overcome these problems and use jclouds with Pod Identities? Is there any plan to add support for them in the future?
Note that we have also opened issue https://github.com/gaul/s3proxy/issues/352 about this on S3Proxy, however the missing functionality seems to be jclouds-related.