Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-4671

When parsing Avro schema, all fields that are not 'top level' are marked as nullable

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.0
    • Component/s: Extensions
    • Labels:
      None

      Description

      From the mailing list, there is an issue with ValidateRecord considering invalid JSON records to be valid when it is missing an 'inner field'. Given the following schema:

      {
        "name": "aRecord",
        "type": "record",
        "namespace": "a",
        "fields": [
          {
            "name": "a",
            "type": {
              "name": "bRecord",
              "type":"record",
              "fields": [
                { "name": "b", "type": "string"}
              ]
            }
          }
      
        ]
      }
      

      ValidateRecord will indicate that the following record is valid:

      {"a":{}}
      

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/2328

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/2328
          Hide
          joewitt Joseph Witt added a comment -

          +1 merged to master. thanks

          Show
          joewitt Joseph Witt added a comment - +1 merged to master. thanks
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f772f2f093de38a97e6a71988628b5fd5aa4139c in nifi's branch refs/heads/master from Mark Payne
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f772f2f ]

          NIFI-4671: This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields

          Signed-off-by: joewitt <joewitt@apache.org>

          Show
          jira-bot ASF subversion and git services added a comment - Commit f772f2f093de38a97e6a71988628b5fd5aa4139c in nifi's branch refs/heads/master from Mark Payne [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f772f2f ] NIFI-4671 : This closes #2328. Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields Signed-off-by: joewitt <joewitt@apache.org>
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user markap14 opened a pull request:

          https://github.com/apache/nifi/pull/2328

          NIFI-4671: Ensure that Avro Schemas that are created properly denote …

          …fields as being nullable iff the schemas says they are, for non-top-level fields

          Thank you for submitting a contribution to Apache NiFi.

          In order to streamline the review of the contribution we ask you
          to ensure the following steps have been taken:

              1. For all changes:
          • [ ] Is there a JIRA ticket associated with this PR? Is it referenced
            in the commit message?
          • [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
          • [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
          • [ ] Is your initial contribution a single, squashed commit?
              1. For code changes:
          • [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
          • [ ] Have you written or updated unit tests to verify your changes?
          • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
          • [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
          • [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
          • [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
              1. For documentation related changes:
          • [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
              1. Note:
                Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/markap14/nifi NIFI-4671

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/2328.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2328


          commit 9197f80bb512450a67b9a9d7a1bab4609abd0e9a
          Author: Mark Payne <markap14@hotmail.com>
          Date: 2017-12-06T16:40:05Z

          NIFI-4671: Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2328 NIFI-4671 : Ensure that Avro Schemas that are created properly denote … …fields as being nullable iff the schemas says they are, for non-top-level fields Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: For all changes: [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? [ ] Is your initial contribution a single, squashed commit? For code changes: [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [ ] Have you written or updated unit tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0] ( http://www.apache.org/legal/resolved.html#category-a)? [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? For documentation related changes: [ ] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4671 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2328.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2328 commit 9197f80bb512450a67b9a9d7a1bab4609abd0e9a Author: Mark Payne <markap14@hotmail.com> Date: 2017-12-06T16:40:05Z NIFI-4671 : Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 closed the pull request at:

          https://github.com/apache/nifi/pull/2323

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/2323
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user markap14 opened a pull request:

          https://github.com/apache/nifi/pull/2323

          NIFI-4671: Ensure that Avro Schemas that are created properly denote …

          …fields as being nullable iff the schemas says they are, for non-top-level fields

          Thank you for submitting a contribution to Apache NiFi.

          In order to streamline the review of the contribution we ask you
          to ensure the following steps have been taken:

              1. For all changes:
          • [ ] Is there a JIRA ticket associated with this PR? Is it referenced
            in the commit message?
          • [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
          • [ ] Has your PR been rebased against the latest commit within the target branch (typically master)?
          • [ ] Is your initial contribution a single, squashed commit?
              1. For code changes:
          • [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
          • [ ] Have you written or updated unit tests to verify your changes?
          • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
          • [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
          • [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
          • [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
              1. For documentation related changes:
          • [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
              1. Note:
                Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/markap14/nifi NIFI-4671

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/2323.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2323


          commit 3c43fc89a97a9770e85825f8533f09388e2c3a59
          Author: Mark Payne <markap14@hotmail.com>
          Date: 2017-12-06T16:40:05Z

          NIFI-4671: Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/2323 NIFI-4671 : Ensure that Avro Schemas that are created properly denote … …fields as being nullable iff the schemas says they are, for non-top-level fields Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: For all changes: [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [ ] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [ ] Has your PR been rebased against the latest commit within the target branch (typically master)? [ ] Is your initial contribution a single, squashed commit? For code changes: [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [ ] Have you written or updated unit tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0] ( http://www.apache.org/legal/resolved.html#category-a)? [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? For documentation related changes: [ ] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-4671 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2323.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2323 commit 3c43fc89a97a9770e85825f8533f09388e2c3a59 Author: Mark Payne <markap14@hotmail.com> Date: 2017-12-06T16:40:05Z NIFI-4671 : Ensure that Avro Schemas that are created properly denote fields as being nullable iff the schemas says they are, for non-top-level fields

            People

            • Assignee:
              markap14 Mark Payne
              Reporter:
              markap14 Mark Payne
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development