Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-11762

TestHCatLoaderEncryption failures when using Hadoop 2.7

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0, 2.0.0
    • Component/s: Shims, Tests
    • Labels:
      None

      Description

      When running TestHCatLoaderEncryption with -Dhadoop23.version=2.7.0, we get the following error during setup():

      testReadDataFromEncryptedHiveTableByPig[5](org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption)  Time elapsed: 3.648 sec  <<< ERROR!
      java.lang.NoSuchMethodError: org.apache.hadoop.hdfs.DFSClient.setKeyProvider(Lorg/apache/hadoop/crypto/key/KeyProviderCryptoExtension;)V
      	at org.apache.hadoop.hive.shims.Hadoop23Shims.getMiniDfs(Hadoop23Shims.java:534)
      	at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.initEncryptionShim(TestHCatLoaderEncryption.java:252)
      	at org.apache.hive.hcatalog.pig.TestHCatLoaderEncryption.setup(TestHCatLoaderEncryption.java:200)
      

      It looks like between Hadoop 2.6 and Hadoop 2.7, the argument to DFSClient.setKeyProvider() changed:

         @VisibleForTesting
      -  public void setKeyProvider(KeyProviderCryptoExtension provider) {
      -    this.provider = provider;
      +  public void setKeyProvider(KeyProvider provider) {
      

        Attachments

        1. HIVE-11762.1.patch
          2 kB
          Jason Dere
        2. HIVE-11762.2.patch
          2 kB
          Jason Dere
        3. HIVE-11762.3.patch
          2 kB
          Jason Dere
        4. HIVE-11762.4.patch
          2 kB
          Jason Dere

          Activity

            People

            • Assignee:
              jdere Jason Dere
              Reporter:
              jdere Jason Dere
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: