Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-7344

Flowfile content viewer appends extra bytes to cipher text

Attach filesAttach ScreenshotAdd voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    Description

      While investigating other issues with encryption services, I had serious difficulty decrypting some content that had been encrypted in NiFi due to cipher pad block errors (almost always indicate the use of the wrong key or the application of a key to the wrong cipher text) and block size errors (indicate invalid cipher text length). 

      Through manual analysis, I found that when data is encrypted with EncryptContent, the data is encrypted correctly and the right bytes are written to the content flowfile. In this case, using the following values, the output should always be 139 bytes. The flowfile size attribute indicates that correctly, and downloading the flowfile content is exactly 139 bytes. However, viewing the content in the embedded content viewer results in 140 bytes, introducing a trailing 0x10 byte that should not be there. 

      So far I have only observed this using the Bcrypt KDF, and not using the PBKDF2 KDF. I will continue to investigate if other KDFs influence this outcome. 

      For plaintext or JSON/XML content, this is a trivial bug. However, when it silently modifies binary cipher text, this is a critical, near blocker, issue. 

       

      Plaintext: This is a plaintext message generated at ${now():format('YYYY-MM-dd HH:mm:ss.SSS Z')}. -> Resolves to "This is a plaintext message generated at 2020-04-08 20:11:44.743 -0700. " which is 72 bytes in UTF-8. 

      KDF: Bcrypt

      Encryption Method: AES-CBC

      Password: thisIsABadPassword

       

      The resulting cipher text should be 139 bytes (29 bytes of Bcrypt salt, 8 bytes of NiFi salt delimiter, 16 bytes of IV, 6 bytes of NiFi IV delimiter, and 80 bytes of cipher text [see below for explanation]). 

       

      This flow is not reproducible on master branch but can be done on NIFI-7122 branch in my repo.  

       

      Cipher text length calculation for AES-CBC:

      int plaintextBlockLength = Math.ceil(plaintext.length() / 16.0)
      boolean plaintextIsExactBlockMultiple = plaintext.length() % 16 == 0
      int cipherTextLength = (plaintextIsExactBlockMultiple ? plaintextBlockLength + 1 : plaintextBlockLength) * 16

      Given the above formula, a plaintext of length 72 requires 5 blocks, but is not an even multiple of 16, so does not require an additional padding block. Rather, 80 bytes (5 * 16) is used. 

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            alopresto Andy LoPresto

            Dates

              Created:
              Updated:

              Slack

                Issue deployment