Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-3859

Provide filtering options in S2SProvenanceReportingTask

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: Extensions
    • Labels:
      None

      Description

      Based on what will be used the S2SProvenanceReportingTask for, it could be nice to provide some filtering options to the user in order to generate flow files only for a subset of the last provenance events.

      Filters could be:

      • Event type
      • Component type
      • Component ID

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user pvillard31 opened a pull request:

          https://github.com/apache/nifi/pull/1777

          NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask

          Thank you for submitting a contribution to Apache NiFi.

          In order to streamline the review of the contribution we ask you
          to ensure the following steps have been taken:

              1. For all changes:
          • [X] Is there a JIRA ticket associated with this PR? Is it referenced
            in the commit message?
          • [X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
          • [X] Has your PR been rebased against the latest commit within the target branch (typically master)?
          • [X] Is your initial contribution a single, squashed commit?
              1. For code changes:
          • [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
          • [X] Have you written or updated unit tests to verify your changes?
          • [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
              1. Note:
                Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/pvillard31/nifi NIFI-3859

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/1777.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1777


          commit 37a0e407cd166c24cd1d143d406319cd00c9ceb2
          Author: Pierre Villard <pierre.villard.fr@gmail.com>
          Date: 2017-05-09T18:44:43Z

          NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user pvillard31 opened a pull request: https://github.com/apache/nifi/pull/1777 NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: For all changes: [X] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [X] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [X] Has your PR been rebased against the latest commit within the target branch (typically master)? [X] Is your initial contribution a single, squashed commit? For code changes: [X] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [X] Have you written or updated unit tests to verify your changes? [X] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/pvillard31/nifi NIFI-3859 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1777.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1777 commit 37a0e407cd166c24cd1d143d406319cd00c9ceb2 Author: Pierre Villard <pierre.villard.fr@gmail.com> Date: 2017-05-09T18:44:43Z NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user ijokarumawak commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1777#discussion_r118419716

          — Diff: nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteProvenanceReportingTask.java —
          @@ -67,14 +73,73 @@

          static final PropertyDescriptor PLATFORM = new PropertyDescriptor.Builder()
          .name("Platform")
          + .displayName("Platform")
          .description("The value to use for the platform field in each provenance event.")
          .required(true)
          .expressionLanguageSupported(true)
          .defaultValue("nifi")
          .addValidator(StandardValidators.NON_EMPTY_VALIDATOR)
          .build();

          + static final PropertyDescriptor FILTER_EVENT_TYPE = new PropertyDescriptor.Builder()
          — End diff –

          These newly added properties are not added to `getSupportedPropertyDescriptors` method. So I was not able to configure filters from NIFI UI.

          Since this is a simple fix, I went ahead and added these filters and able to confirm provenance events are filtered. LGTM, +1. I will merge this into master. Thanks @pvillard31 !

          Show
          githubbot ASF GitHub Bot added a comment - Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1777#discussion_r118419716 — Diff: nifi-nar-bundles/nifi-site-to-site-reporting-bundle/nifi-site-to-site-reporting-task/src/main/java/org/apache/nifi/reporting/SiteToSiteProvenanceReportingTask.java — @@ -67,14 +73,73 @@ static final PropertyDescriptor PLATFORM = new PropertyDescriptor.Builder() .name("Platform") + .displayName("Platform") .description("The value to use for the platform field in each provenance event.") .required(true) .expressionLanguageSupported(true) .defaultValue("nifi") .addValidator(StandardValidators.NON_EMPTY_VALIDATOR) .build(); + static final PropertyDescriptor FILTER_EVENT_TYPE = new PropertyDescriptor.Builder() — End diff – These newly added properties are not added to `getSupportedPropertyDescriptors` method. So I was not able to configure filters from NIFI UI. Since this is a simple fix, I went ahead and added these filters and able to confirm provenance events are filtered. LGTM, +1. I will merge this into master. Thanks @pvillard31 !
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit b6eb0ac0fb382b5cc2e3f60196e36835437883f3 in nifi's branch refs/heads/master from Pierre Villard
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=b6eb0ac ]

          NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask

          This closes #1777.

          Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>

          Show
          jira-bot ASF subversion and git services added a comment - Commit b6eb0ac0fb382b5cc2e3f60196e36835437883f3 in nifi's branch refs/heads/master from Pierre Villard [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=b6eb0ac ] NIFI-3859 - Provide filtering options in S2SProvenanceReportingTask This closes #1777. Signed-off-by: Koji Kawamura <ijokarumawak@apache.org>
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/1777

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1777

            People

            • Assignee:
              pvillard Pierre Villard
              Reporter:
              pvillard Pierre Villard
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development