Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-10235

Provenance replay fails when repository encryption is enabled

    XMLWordPrintableJSON

Details

    Description

      Problem summary

      When repository encryption is enabled, replaying a DROP provenance record fails, with the following error appearing in the logs:

      org.apache.nifi.processor.exception.FlowFileAccessException: Failed to export StandardFlowFileRecord[uuid=df985fc5-23da-4094-8783-2e0186bcb92d,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1657864218374-23, container=default, section=23], offset=379, length=1048576],offset=0,name=b29633c4-324e-42fe-b3e8-1ea455fc3650,size=1048576] to /opt/nifi/nifi-current/data/store/.b29633c4-324e-42fe-b3e8-1ea455fc3650 due to java.io.EOFException: Attempted to copy 1048576 bytes but only 1048197 bytes were available

       
      I've observed that the difference between the sizes mentioned in the log is always 379 bytes, regardless of the length of the input file.
       
      With repository encryption disabled, provenance replay works as expected.

      Configuration

      1. NiFi v1.16.3 running as a three-node cluster in Kubernetes.
      2. Each node has up to 8GB memory and 4 CPUs available to it.
      3. Testing has included both NFS and ephemeral (emptyDir) storage.
      4. The encryption key was generated by the following command, using the same JDK version:
        1. keytool -genseckey -alias key-1 -keyalg AES -keysize 256 -keystore repository.p12 -storetype PKCS12

      nifi.properties

      nifi.repository.encryption.protocol.version=1
      nifi.repository.encryption.key.id=key-1
      nifi.repository.encryption.key.provider=KEYSTORE
      nifi.repository.encryption.key.provider.keystore.location=conf/repository.p12
      nifi.repository.encryption.key.provider.keystore.password=<password>

      Processor group

      GenerateFlowFile processor generating 1MB random files every second to a PutFile processor. Have also tested with InvokeHTTP.

      Other comments

      With repository encryption enabled, I am able to download files via the provenance UI (suggesting that encryption/decryption works). The processor group also performs all other actions as expected.

      Not having the ability to replay provenance records is a blocker for our deployment, which requires data to be encrypted at rest and in transit.

      Attachments

        1. error.log
          3 kB
          Peter Kimberley
        2. error-base-install.log
          5 kB
          Peter Kimberley
        3. NiFi_Flow.json
          5 kB
          Peter Kimberley

        Issue Links

          Activity

            People

              exceptionfactory David Handermann
              p-kimberley Peter Kimberley
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 1h
                  1h