Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-5420

Allow StandardProcessSession to calculate duration for provenance events

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.8.0
    • Component/s: Core Framework
    • Labels:
      None

      Description

      Besides some automatic provenance events being emitted by the framework, the onus is on each processor to register provenance events and to provide a duration for the amount of time spent on the operation that results in the provenance event(s). However, supplying the duration is not mandatory, and if not provided, a value of -1 is stored to indicate that the duration is unknown.

      Unfortunately there are many processors that do not track the operation time and thus do not store it in the provenance event. However, the time from when a flow file is fetched or created in a session to the time the provenance event is recorded is a legitimate measure of the time the flow file spent "active" in the session, which is the same as the duration of a provenance event.

      This Jira proposes to record the "start" time of each flow file (more specifically, it's StandardRepositoryRecord), and upon session.commit(), record the duration of the flow file while active in the session into the provenance event(s), only if the duration has not already been supplied to the provenance events.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                mattyb149 Matt Burgess
                Reporter:
                mattyb149 Matt Burgess
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: