Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-4539

ExtractGrok - Add support returning only named captures

    Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.4.0
    • Fix Version/s: None
    • Component/s: Extensions
    • Labels:
      None

      Description

      ExtractGrok support named captures only option.
      Currently, ExtractGrok returns all matches for a grok pattern. In some case, this is verbose.

      Following example parse apache common access log.

      83.149.9.216 - - [17/May/2015:10:05:03 +0000] "GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1" 200 203023 "http://semicomplete.com/presentations/logstash-monitorama-2013/" "Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/32.0.1700.77 Safari/537.36
      

      Disable named captures only

      {
        "grok.auth": "-",
        "grok.timestamp": "17/May/2015:10:05:03 +0000",
        "grok.httpversion": "1.1",
        "grok.HOUR": "10",
        "grok.ident": "-",
        "grok.SECOND": "03",
        "grok.HTTPD_COMMONLOG": "83.149.9.216 - - [17/May/2015:10:05:03 +0000] \"GET /presentations/logstash-monitorama-2013/images/kibana-search.png HTTP/1.1\" 200 203023",
        "grok.USERNAME": "[-, -]",
        "grok.IP": "83.149.9.216",
        "grok.clientip": "83.149.9.216",
        "grok.verb": "GET",
        "grok.EMAILADDRESS": "[null, null]",
        "grok.request": "/presentations/logstash-monitorama-2013/images/kibana-search.png",
        "grok.EMAILLOCALPART": "[null, null]",
        "grok.INT": "+0000",
        "grok.BASE10NUM": "[1.1, 200, 203023]",
        "grok.YEAR": "2015",
        "grok.IPV4": "83.149.9.216",
        "grok.MINUTE": "05",
        "grok.HOSTNAME": "[null, null, null]",
        "grok.USER": "[-, -]",
        "grok.response": "200",
        "grok.bytes": "203023",
        "grok.TIME": "10:05:03",
        "grok.MONTH": "May",
        "grok.MONTHDAY": "17"
      }
      

      Enable named captures only

      {
        "grok.request": "/presentations/logstash-monitorama-2013/images/kibana-search.png",
        "grok.auth": "-",
        "grok.ident": "-",
        "grok.timestamp": "17/May/2015:10:05:03 +0000",
        "grok.httpversion": "1.1",
        "grok.clientip": "83.149.9.216",
        "grok.response": "200",
        "grok.bytes": "203023",
        "grok.verb": "GET"
      }
      

        Issue Links

          Activity

          Show
          Norito Agetsuma Norito Agetsuma added a comment - https://github.com/apache/nifi/pull/2233
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user n-agetsu opened a pull request:

          https://github.com/apache/nifi/pull/2233

          NIFI-4539 ExtractGrok - Add support returning only named captures

          Thank you for submitting a contribution to Apache NiFi.

          In order to streamline the review of the contribution we ask you
          to ensure the following steps have been taken:

              1. For all changes:
          • [x] Is there a JIRA ticket associated with this PR? Is it referenced
            in the commit message?
          • [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
          • [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
          • [x] Is your initial contribution a single, squashed commit?
              1. For code changes:
          • [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
          • [x] Have you written or updated unit tests to verify your changes?
          • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
          • [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
          • [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
          • [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
              1. For documentation related changes:
          • [x] Have you ensured that format looks appropriate for the output in which it is rendered?
              1. Note:
                Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/n-agetsu/nifi NIFI-4539

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/2233.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2233


          commit 51ad5d4c6c94f5ba0c7d462f2200a8fd25c5e43e
          Author: Norito Agetsuma <norito.agetsuma@gmail.com>
          Date: 2017-10-28T05:20:34Z

          NIFI-4539 ExtractGrok - Add support returning only named captures


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user n-agetsu opened a pull request: https://github.com/apache/nifi/pull/2233 NIFI-4539 ExtractGrok - Add support returning only named captures Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: For all changes: [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [x] Has your PR been rebased against the latest commit within the target branch (typically master)? [x] Is your initial contribution a single, squashed commit? For code changes: [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [x] Have you written or updated unit tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0] ( http://www.apache.org/legal/resolved.html#category-a)? [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? For documentation related changes: [x] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/n-agetsu/nifi NIFI-4539 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2233.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2233 commit 51ad5d4c6c94f5ba0c7d462f2200a8fd25c5e43e Author: Norito Agetsuma <norito.agetsuma@gmail.com> Date: 2017-10-28T05:20:34Z NIFI-4539 ExtractGrok - Add support returning only named captures

            People

            • Assignee:
              Unassigned
              Reporter:
              Norito Agetsuma Norito Agetsuma
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development