Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11762

TestHCatLoaderEncryption failures when using Hadoop 2.7

Log workAgile BoardRank to TopRank to BottomBulk Copy AttachmentsBulk Move AttachmentsVotersWatch issueWatchersCreate sub-taskConvert to sub-taskMoveLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 1.3.0, 2.0.0
    • Shims, Tests
    • None

    Description

      When running TestHCatLoaderEncryption with -Dhadoop23.version=2.7.0, we get the following error during setup():

      testReadDataFromEncryptedHiveTableByPig[5](org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption)  Time elapsed: 3.648 sec  <<< ERROR!
      java.lang.NoSuchMethodError: org.apache.hadoop.hdfs.DFSClient.setKeyProvider(Lorg/apache/hadoop/crypto/key/KeyProviderCryptoExtension;)V
      	at org.apache.hadoop.hive.shims.Hadoop23Shims.getMiniDfs(Hadoop23Shims.java:534)
      	at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.initEncryptionShim(TestHCatLoaderEncryption.java:252)
      	at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:200)
      

      It looks like between Hadoop 2.6 and Hadoop 2.7, the argument to DFSClient.setKeyProvider() changed:

         @VisibleForTesting
      -  public void setKeyProvider(KeyProviderCryptoExtension provider) {
      -    this.provider = provider;
      +  public void setKeyProvider(KeyProvider provider) {
      

      Attachments

        1. HIVE-11762.4.patch
          2 kB
          Jason Dere
        2. HIVE-11762.3.patch
          2 kB
          Jason Dere
        3. HIVE-11762.2.patch
          2 kB
          Jason Dere
        4. HIVE-11762.1.patch
          2 kB
          Jason Dere

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jdere Jason Dere Assign to me
            jdere Jason Dere
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment