Major Description
UC-5: S3 Integration
While the Knox WebHDFS integration may still work for many cloud deployments, it does seem like a gap that there is no way to move files in and out of S3 or other cloud storage mechanisms through Knox.
We can actually combine UC-2 above to acquire temporary credentials on behalf of the authenticated users. We would request the IAM role and permissions that are appropriate for the user and their group memberships in order to access buckets protected with IAM roles. We could also combine with UC-4 above to have encrypted files put into S3 that will only be able to be decrypted on-prem.
It would require Knox to be granted permission in a given cloud deployment to make STS calls and may require AWS credentials for the Knox user to be an IAM role. We may also be able to assumeRole to the needed role for STS access.
It will also require a Jersey service hosted in Knox to put files into S3 (KnoxS3?) or we can create a pluggable backend and make it a more generic object store API.