[NIFI-3116] Remove Jasypt library

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.1.0
• Fix Version/s: 1.4.0
• Component/s: Core Framework
• Labels: encryption kdf pbe security
• Assignee: Andy LoPresto
• Reporter: Andy LoPresto
• Created: 29/Nov/16 03:57
• Updated: 26/Sep/17 17:27
• Resolved: 26/Sep/17 14:57

Description

The Jasypt library is used internally by NiFi for String encryption operations (specifically password-based encryption (PBE) in EncryptContent and sensitive processor property protection). I feel there are a number of reasons to remove this library from NiFi and provide centralized symmetric encryption operations using Java cryptographic primitives (and BouncyCastle features where necessary).

• The library was last updated February 25, 2014. For comparison, BouncyCastle has been updated 5 times since then
• StandardPBEStringEncryptor, the high-level class wrapped by NiFi's StringEncryptor is final. This makes it, and features relying on it, difficult to test in isolation
• Jasypt encapsulates many decisions about Cipher configuration, specifically salt-generation strategy. This can be a valuable feature for pluggable libraries, but is less than ideal when dealing with encryption and key derivation, which are in constant struggle with evolving attacks and improving hardware. There are hard-coded constants which are not compatible with better decisions available now (i.e. requiring custom implementations of the SaltGenerator interface to provide new derivations). The existence of these values was opaque to NiFi and led to serious compatibility issues NIFI-1259, NIFI-1257, NIFI-1242, NIFI-1463, NIFI-1465, NIFI-3024
• StringEncryptor, the NiFi class wrapping StandardPBEStringEncryptor is also final and does not expose methods to instantiate it with only the relevant values (i.e. algorithm, provider, and password) but rather requires an entire NiFiProperties instance.
• StringEncryptor.createEncryptor() performs an unnecessary "validation check" on instantiation, which was one cause of reported issues where a secure node/cluster blocks on startup on VMs due to lack of entropy in /dev/random
• The use of custom salts with PBE means that the internal Cipher object must be re-created and initialized and the key re-derived from the password on every decryption call. Symmetric keyed encryption with a strong KDF (order of magnitude higher iterations of a stronger algorithm) and unique initialization vector (IV) values would be substantially more resistant to brute force attacks and yet more performant at scale.

I have already implemented backwards-compatible code to perform the actions of symmetric key encryption using keys derived from passwords in both the ConfigEncryptionTool and OpenSSLPKCS5CipherProvider and NiFiLegacyCipherProvider classes, which empirical tests confirm are compatible with the Jasypt output.

Additional research on some underlying/related issues:

• Why does Java allow AES-256 bit encryption on systems without JCE unlimited strength policies if using PBE?
• How To Decrypt OpenSSL-encrypted Data In Apache NiFi
• [e-mail] "Passwords in EncryptContent"

Issue Links

• is related to: NIFI-3313 First deployment of NiFi can hang on VMs without sufficient entropy if using /dev/random
• relates to: NIFI-4237 EncryptionOperationNotPossibleException in nifi-bootstrap.log might suggest underlying cause
• relates to: NIFI-1465 Upgrade encryption of sensitive properties
• links to: GitHub Pull Request #2108