Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-6040

ExecuteSQL can't perform batch loading with an input file

    XMLWordPrintableJSON

    Details

    • Flags:
      Important

      Description

      Hello!

      TC to reproduce:

      1) Create GenerateFlowFile with the next Custom Text:

      select 1 from dual
      union all
      select 2 from dual
      union all
      select 3 from dual
      

      2) Create ExecuteSQL. Set "Output Batch Size" = 1.
      3) Start

      AR:

      ER:

      3 flowfiles in the "success" connection.

      Root Cause:

      if (outputBatchSize > 0 && resultSetFlowFiles.size() >= outputBatchSize) {
          session.transfer(resultSetFlowFiles, REL_SUCCESS);
          session.commit();
          resultSetFlowFiles.clear();
      }
      

      Commit method has the next requirement:
      Commits the current session ensuring all operations against FlowFiles within this session are atomically persisted. All FlowFiles operated on within this session must be accounted for by transfer or removal or the commit will fail.

      At the time of any commit, an input flowfile (fileToProcess) still exists and has not been accounted for by transfer or removal.

        Attachments

        1. image-2019-02-15-18-13-39-190.png
          360 kB
          Artem Anokhin

          Issue Links

            Activity

              People

              • Assignee:
                mattyb149 Matt Burgess
                Reporter:
                JFeoks Artem Anokhin
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m