Major Description
A pre-requisite for getting HDFS encryption in branch-2 is KMS, we need to merge all related JIRAs:
052932e7299ff64d36287b368f94ccf8698d5c9d HADOOP-10141. Create KeyProvider API to separate encryption key storage from the applications. (omalley) b72026617b038f588581d43c323718fe8120b400 HADOOP-10201. Add listing to KeyProvider API. (Larry McCay via omalley) 4a178b6736d54e1b1940babd7cbda34921957d01 HADOOP-10177. Create CLI tools for managing keys. (Larry McCay via omalley) 0cf6ccf606fceb6c06f35d72b2c2b679d71ad96c HADOOP-10237. JavaKeyStoreProvider needs to set keystore permissions correctly. (Larry McCay via omalley) 56d349b81d24ef1421ffcdfb822a8fe122f05c80 HADOOP-10432. Refactor SSLFactory to expose static method to determine HostnameVerifier. (tucu) 0d66663cb277937eb7ec6a281dc7f236efe387fd HADOOP-10429. KeyStores should have methods to generate the materials themselves, KeyShell should use them. (tucu) d9c1c42fdfddb810ebe2ec151f751d05e987f25e HADOOP-10427. KeyProvider implementations should be thread safe. (tucu) 98be41ff908acd2fa55c0b302c8a3def55987e41 HADOOP-10428. JavaKeyStoreProvider should accept keystore password via configuration falling back to ENV VAR. (tucu) b2b05181682c2a55f5ed1cfa2c44f3390eebd5c4 HADOOP-10244. TestKeyShell improperly tests the results of delete (Larry McCay via omalley) 83f057e8e1d16949b94fe2e99f4232ced8156e6a HADOOP-10430. KeyProvider Metadata should have an optional description, there should be a method to retrieve the metadata from all keys. (tucu) f6f52ca1c2df57d13fa596e074accc0f3549ff58 HADOOP-10431. Change visibility of KeyStore.Options getter methods to public. (tucu) 05e59fd8058f21a52d4a268af3a189c89ebad2fe HADOOP-10534. KeyProvider getKeysMetadata should take a list of names rather than returning all keys. (omalley) 16be41a63e4b3bd79b1cee4edce6df374666ca58 HADOOP-10433. Key Management Server based on KeyProvider API. (tucu) 4bcaa45a2ea36fb440069c7a458cdc225cb862ca HADOOP-10583. bin/hadoop key throws NPE with no args and assorted other fixups. (clamb via tucu) 1727e235c3d3317b2ac6d7c25ea01505853653ca HADOOP-10586. KeyShell doesn't allow setting Options via CLI. (clamb via tucu) 6b410f3b2e185fca963c7db664395e97d76cd6ee HADOOP-10645. TestKMS fails because race condition writing acl files. (tucu) 7868054902590af6dbda941f2cc8324267c8bef8 HADOOP-10611. KMS, keyVersion name should not be assumed to be keyName@versionNumber. (tucu) 725f087f3f2fc31190810344d0e508e34b4a126e HADOOP-10607. Create API to separate credential/password storage from applications. (Larry McCay via omalley) 097254f094b004404ba4754f97f906f46a12b0e4 HADOOP-10696. Add optional attributes to KeyProvider Options and Metadata. (tucu) a283b91add9e9230b9597fd33355822517a1852e HADOOP-10695. KMSClientProvider should respect a configurable timeout. (yoderme via tucu) 6cef126f29673704c345c52995890ff48395ec1a HADOOP-10757. KeyProvider KeyVersion should provide the key name. (asuresh via tucu) 9b7a1cb122c6a6041e718986085ec7f6bab422c4 HADOOP-10719. Add generateEncryptedKey and decryptEncryptedKey methods to KeyProvider. (asuresh via tucu) 9c03a4b321db7950d5652ba03022f9ee3ebd2d6f HADOOP-10769. Create KeyProvider extension to handle delegation tokens. Contributed by Arun Suresh. db91ab3d02fddfd325fd308e46f65075c2c6cd93 HADOOP-10812. Delegate KeyProviderExtension#toString to underlying KeyProvider. (wang) 7c7911bbd63d30932df71af536f45c20adba88ff HADOOP-10736. Add key attributes to the key shell. Contributed by Mike Yoder. cfb5943d356fef911f424ed8250a9c02b706ecc6 HADOOP-10824. Refactor KMSACLs to avoid locking. (Benoy Antony via umamahesh) 6b9b985233c293d22f89a4deadf871230f09d7ed HADOOP-10816. KeyShell returns -1 on error to the shell, should be 1. (Mike Yoder via wang) ceea01cff5762115c58817ab696cd11641bc9a98 HADOOP-10841. EncryptedKeyVersion should have a key name property. (asuresh via tucu) 468a4fc00921ea7bc61bb60666e9352b0ad3928b HADOOP-10842. CryptoExtension generateEncryptedKey method should receive the key name. (asuresh via tucu) c6d60c6db8b22d6dc45e63073bc5bb52dc041a8c HADOOP-10750. KMSKeyProviderCache should be in hadoop-common. (asuresh via tucu) c3eca9f2504ed619a3edcf3d3eafc286133911d0 HADOOP-10720. KMS: Implement generateEncryptedKey and decryptEncryptedKey in the REST API. (asuresh via tucu) 6ae46e601290a094019fdd8e241a90a6f269203c HADOOP-10826. Iteration on KeyProviderFactory.serviceLoader is thread-unsafe. (benoyantony viat tucu) 22bbb1e1b1ad076cb2cac22b7863904aea903586 HADOOP-10881. Clarify usage of encryption and encrypted encryption key in KeyProviderCryptoExtension. (wang) 8eafb8915177261d6560c365c5cac6f7dad12e55 HADOOP-10891. Add EncryptedKeyVersion factory method to KeyProviderCryptoExtension. (wang) cae52dee46a57da40a811129781a3664beb0fe42 HADOOP-10756. KMS audit log should consolidate successful similar requests. (asuresh via tucu) 9704e448046a95949d6da6c894f729130821f88b HADOOP-10793. KeyShell args should use single-dash style. (wang) 13e092f3ecfb11e9bc33cae7f81768f393f9ac64 HADOOP-10920. site plugin couldn't parse hadoop-kms index.apt.vm. (Akira Ajisaka via wang) 362bc16eaa7d83a3ef9dde5e6c69f21f753b8a80 HADOOP-10937. Need to set version name correctly before decrypting EEK. Contributed by Arun Suresh. 66af8b0ed51f082889be3d39f63e28f5920e5cb6 HADOOP-10936. Change default KeyProvider bitlength to 128. (wang) e1eb546528ee4d5c1c44f8d785bf0c0378090645 HADOOP-10918. JMXJsonServlet fails when used within Tomcat. (tucu) b4706add323b7fb195844d4b4ec10d445f7122fd HADOOP-10939. Fix TestKeyProviderFactory testcases to use default 128 bit length keys. Contributed by Arun Suresh. 75abed80c6314623e4eb842d003c6613e493a16b HADOOP-10862. Miscellaneous trivial corrections to KMS classes. (asuresh via tucu) 0d2970300a4074dbc448d6d79946444afa6e66d9 HADOOP-10224. JavaKeyStoreProvider has to protect against corrupting underlying store. (asuresh via tucu) d8663c28e0f26af9b34fdead2fe4cd7ed628e2e2 HADOOP-10770. KMS add delegation token support. (tucu) 859fe45e4e22d96f22dd35649cd25ab7c94ba444 HADOOP-10967. Improve DefaultCryptoExtension#generateEncryptedKey performance. (hitliuyi via tucu) 9e87d275322482133054454bea8c34d49703105f HADOOP-10698. KMS, add proxyuser support. (tucu) 45b61bfa07007e3807ee8ee5ed36c058f8042983 HADOOP-10488. TestKeyProviderFactory fails randomly. (tucu)