Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11329

Add JAVA_LIBRARY_PATH to KMS startup options

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.7.0
    • Component/s: kms, security
    • Labels:
      None

      Description

      Currently, HADOOP_HOME isn't part of the start up options of KMS. If I add the the following configuration to core-site.xml of kms,

       <property>
        <name>hadoop.security.crypto.codec.classes.aes.ctr.nopadding</name>
        <value>org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec</value>
       </property>
      

      kms server will throw the following exception when receive "generateEncryptedKey" request

      2014-11-24 10:23:18,189 DEBUG org.apache.hadoop.crypto.OpensslCipher: Failed to load OpenSSL Cipher.
      java.lang.UnsatisfiedLinkError: org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl()Z
              at org.apache.hadoop.util.NativeCodeLoader.buildSupportsOpenssl(Native Method)
              at org.apache.hadoop.crypto.OpensslCipher.<clinit>(OpensslCipher.java:85)
              at org.apache.hadoop.crypto.OpensslAesCtrCryptoCodec.<init>(OpensslAesCtrCryptoCodec.java:50)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
              at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
              at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
              at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
              at org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:129)
              at org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:67)
              at org.apache.hadoop.crypto.CryptoCodec.getInstance(CryptoCodec.java:100)
              at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension$DefaultCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:256)
              at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:371)
              at org.apache.hadoop.crypto.key.kms.server.EagerKeyGeneratorKeyProviderCryptoExtension$CryptoExtension$EncryptedQueueRefiller.fillQueueForKey(EagerKeyGeneratorKeyProviderCryptoExtension.java:77)
              at org.apache.hadoop.crypto.key.kms.ValueQueue$1.load(ValueQueue.java:181)
              at org.apache.hadoop.crypto.key.kms.ValueQueue$1.load(ValueQueue.java:175)
              at com.google.common.cache.LocalCache$LoadingValueReference.loadFuture(LocalCache.java:3568)
              at com.google.common.cache.LocalCache$Segment.loadSync(LocalCache.java:2350)
              at com.google.common.cache.LocalCache$Segment.lockedGetOrLoad(LocalCache.java:2313)
              at com.google.common.cache.LocalCache$Segment.get(LocalCache.java:2228)
              at com.google.common.cache.LocalCache.get(LocalCache.java:3965)
              at com.google.common.cache.LocalCache.getOrLoad(LocalCache.java:3969)
              at com.google.common.cache.LocalCache$LocalManualCache.get(LocalCache.java:4829)
              at org.apache.hadoop.crypto.key.kms.ValueQueue.getAtMost(ValueQueue.java:256)
              at org.apache.hadoop.crypto.key.kms.ValueQueue.getNext(ValueQueue.java:226)
              at org.apache.hadoop.crypto.key.kms.server.EagerKeyGeneratorKeyProviderCryptoExtension$CryptoExtension.generateEncryptedKey(EagerKeyGeneratorKeyProviderCryptoExtension.java:126)
              at org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.generateEncryptedKey(KeyProviderCryptoExtension.java:371)
              at org.apache.hadoop.crypto.key.kms.server.KeyAuthorizationKeyProvider.generateEncryptedKey(KeyAuthorizationKeyProvider.java:192)
              at org.apache.hadoop.crypto.key.kms.server.KMS$9.run(KMS.java:379)
              at org.apache.hadoop.crypto.key.kms.server.KMS$9.run(KMS.java:375
      

      The reason is that it cannot find libhadoop.so. This will prevent KMS to response to "generateEncryptedKey" requests.

        Attachments

        1. HADOOP-11329.9.patch
          3 kB
          Arun Suresh
        2. HADOOP-11329.8.patch
          3 kB
          Arun Suresh
        3. HADOOP-11329.7.patch
          3 kB
          Arun Suresh
        4. HADOOP-11329.6.patch
          3 kB
          Arun Suresh
        5. HADOOP-11329.5.patch
          4 kB
          Arun Suresh
        6. HADOOP-11329.4.patch
          4 kB
          Arun Suresh
        7. HADOOP-11329.3.patch
          2 kB
          Arun Suresh
        8. HADOOP-11329.2.patch
          2 kB
          Arun Suresh
        9. HADOOP-11329.1.patch
          1 kB
          Arun Suresh

          Activity

            People

            • Assignee:
              asuresh Arun Suresh
              Reporter:
              dian.fu Dian Fu
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: