Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-6662

Access AWS access key ID and secret access key using Credential Provider API for S3 storage plugin

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.15.0
    • Component/s: None

      Description

      Hadoop provides [CredentialProvider API|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/CredentialProviderAPI.html] which allows passwords and other sensitive secrets to be stored in an external provider rather than in configuration files in plaintext.

      Currently S3 storage plugin is accessing passwords, namely 'fs.s3a.access.key' and 'fs.s3a.secret.key', stored in clear text in Configuration with get() method. To give users an ability to remove clear text passwords for S3 from configuration files Configuration.getPassword() method should be used, given they configure 'hadoop.security.credential.provider.path' property which points to a file containing encrypted passwords instead of configuring two aforementioned properties.

      By using this approach, credential providers will be checked first and if the secret is not provided or providers are not configured there will be a fallback to secrets configured in clear text (unless 'hadoop.security.credential.clear-text-fallback' is configured to be "false"), thus making new change backwards-compatible.

        Attachments

          Activity

            People

            • Assignee:
              KazydubB Bohdan Kazydub
              Reporter:
              KazydubB Bohdan Kazydub
              Reviewer:
              Arina Ielchiieva
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: