Uploaded image for project: 'Sqoop (Retired)'
  1. Sqoop (Retired)
  2. SQOOP-1471

Use Hadoop CredentialProvider API to encyrpt passwords at rest

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.4.4, 1.4.5
    • 1.4.6
    • None
    • None
    • Hadoop 2.6.0+

    Description

      HADOOP-10607 introduces an API to isolate password Storage from applications. This can be used to provide the DB password in a secure fashion to Sqoop users.

      The DB passwords can be assigned an alias and stored in a keystore and the alias can be provided to Sqoop.
      Option to provide the password on the command line (using --password-alias) option can be introduced as well as enhance the secure file option that we currently have to provide a new password loader that resolves the alias from keystore.

      We need to detect the availability of the Credential Provider feature and use it and fail gracefully otherwise

      Attachments

        1. SQOOP-1471.diff.4
          24 kB
          Venkat Ranganathan
        2. SQOOP-1471.diff.3
          22 kB
          Venkat Ranganathan
        3. SQOOP-1471.diff.2
          22 kB
          Venkat Ranganathan
        4. SQOOP-1471.diff
          21 kB
          Venkat Ranganathan

        Issue Links

          Activity

            Uploading new patch that is available in RB

            venkatnrangan Venkat Ranganathan added a comment - Uploading new patch that is available in RB

            abec Please let me know if you have additional review items. Thanks

            venkatnrangan Venkat Ranganathan added a comment - abec Please let me know if you have additional review items. Thanks

            Uploading new patch

            venkatnrangan Venkat Ranganathan added a comment - Uploading new patch

            Commit 65a9340fc22fcc3c1d2c64b3976994aa1c419b8f in sqoop's branch refs/heads/trunk from abec
            [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=65a9340 ]

            SQOOP-1471: Use Hadoop CredentialProvider API to encyrpt passwords at rest

            (Venkat Ranganathan via Abraham Elmahrek)

            jira-bot ASF subversion and git services added a comment - Commit 65a9340fc22fcc3c1d2c64b3976994aa1c419b8f in sqoop's branch refs/heads/trunk from abec [ https://git-wip-us.apache.org/repos/asf?p=sqoop.git;h=65a9340 ] SQOOP-1471 : Use Hadoop CredentialProvider API to encyrpt passwords at rest (Venkat Ranganathan via Abraham Elmahrek)

            +1 thanks venkatnrangan!

            abec Abraham Elmahrek added a comment - +1 thanks venkatnrangan !
            hudson Hudson added a comment -

            FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #938 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/938/)
            SQOOP-1471: Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f)

            • src/java/org/apache/sqoop/util/password/FilePasswordLoader.java
            • src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java
            • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
            • src/java/org/apache/sqoop/SqoopOptions.java
            • src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java
            • src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java
            • src/docs/user/connecting.txt
            hudson Hudson added a comment - FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop20 #938 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop20/938/ ) SQOOP-1471 : Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f ) src/java/org/apache/sqoop/util/password/FilePasswordLoader.java src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java src/docs/user/connecting.txt

            Thanks for reviewing and committing it abec Very much appreciated.

            venkatnrangan Venkat Ranganathan added a comment - Thanks for reviewing and committing it abec Very much appreciated.
            hudson Hudson added a comment -

            SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1141 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1141/)
            SQOOP-1471: Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f)

            • src/java/org/apache/sqoop/util/password/FilePasswordLoader.java
            • src/docs/user/connecting.txt
            • src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java
            • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
            • src/java/org/apache/sqoop/SqoopOptions.java
            • src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java
            • src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java
            hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop23 #1141 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop23/1141/ ) SQOOP-1471 : Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f ) src/java/org/apache/sqoop/util/password/FilePasswordLoader.java src/docs/user/connecting.txt src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java

            Np!

            The test failures are around the mainframe code base. I've created SQOOP-1570 to track this.

            abec Abraham Elmahrek added a comment - Np! The test failures are around the mainframe code base. I've created SQOOP-1570 to track this.
            hudson Hudson added a comment -

            FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #903 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/903/)
            SQOOP-1471: Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f)

            • src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java
            • src/java/org/apache/sqoop/SqoopOptions.java
            • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
            • src/java/org/apache/sqoop/util/password/FilePasswordLoader.java
            • src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java
            • src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java
            • src/docs/user/connecting.txt
            hudson Hudson added a comment - FAILURE: Integrated in Sqoop-ant-jdk-1.6-hadoop100 #903 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop100/903/ ) SQOOP-1471 : Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f ) src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java src/java/org/apache/sqoop/SqoopOptions.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/util/password/FilePasswordLoader.java src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java src/docs/user/connecting.txt
            hudson Hudson added a comment -

            SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #945 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/945/)
            SQOOP-1471: Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f)

            • src/java/org/apache/sqoop/SqoopOptions.java
            • src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java
            • src/java/org/apache/sqoop/tool/BaseSqoopTool.java
            • src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java
            • src/java/org/apache/sqoop/util/password/FilePasswordLoader.java
            • src/docs/user/connecting.txt
            • src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java
            hudson Hudson added a comment - SUCCESS: Integrated in Sqoop-ant-jdk-1.6-hadoop200 #945 (See https://builds.apache.org/job/Sqoop-ant-jdk-1.6-hadoop200/945/ ) SQOOP-1471 : Use Hadoop CredentialProvider API to encyrpt passwords at rest (abraham: https://git-wip-us.apache.org/repos/asf?p=sqoop.git&a=commit&h=65a9340fc22fcc3c1d2c64b3976994aa1c419b8f ) src/java/org/apache/sqoop/SqoopOptions.java src/test/org/apache/sqoop/credentials/TestPassingSecurePassword.java src/java/org/apache/sqoop/tool/BaseSqoopTool.java src/java/org/apache/sqoop/util/password/CredentialProviderHelper.java src/java/org/apache/sqoop/util/password/FilePasswordLoader.java src/docs/user/connecting.txt src/java/org/apache/sqoop/util/password/CredentialProviderPasswordLoader.java

            People

              venkatnrangan Venkat Ranganathan
              venkatnrangan Venkat Ranganathan
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: