Uploaded image for project: 'Avro'
  1. Avro
  2. AVRO-2075

Allow SchemaCompatibility to report possibly lossy conversions

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Minor
    • Resolution: Unresolved
    • Affects Version/s: 1.7.7, 1.8.2
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Environment:

      Java

      Description

      It is stated in the Avro spec that int and long values are promotable to floats and doubles.
      However, numeric promotions to floats are lossy (losing precision), as is long promotion to double.
      It is suggested that the SchemaCompatibility class is updated to be able to flag conversions that have the possibility to be lossy as errors. The attached patch does just that, by adding a new boolean flag (allowDataLoss), preserving backwards compatibility by defaulting this flag to true.
      Testcases illustrating the problem has been added to the unit test class TestReadingWritingDataInEvolvedSchemas

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user epkanol opened a pull request:

          https://github.com/apache/avro/pull/246

          AVRO-2075: Add option to report possible data loss in SchemaCompatibi…

          Same patch as in the JIRA

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/epkanol/avro-1 AVRO-2075

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/avro/pull/246.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #246


          commit bfaacb70a788a3bcf1639ca592ea64ef2d21dfeb
          Author: Anders Sundelin <anders.sundelin@ericsson.com>
          Date: 2017-09-10T17:37:02Z

          AVRO-2075: Add option to report possible data loss in SchemaCompatibility


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user epkanol opened a pull request: https://github.com/apache/avro/pull/246 AVRO-2075 : Add option to report possible data loss in SchemaCompatibi… Same patch as in the JIRA You can merge this pull request into a Git repository by running: $ git pull https://github.com/epkanol/avro-1 AVRO-2075 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/avro/pull/246.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #246 commit bfaacb70a788a3bcf1639ca592ea64ef2d21dfeb Author: Anders Sundelin <anders.sundelin@ericsson.com> Date: 2017-09-10T17:37:02Z AVRO-2075 : Add option to report possible data loss in SchemaCompatibility
          Hide
          epkanol Anders Sundelin added a comment -
          Show
          epkanol Anders Sundelin added a comment - Relevant PR: https://github.com/apache/avro/pull/246
          Hide
          epkanol Anders Sundelin added a comment -

          Ping Elliot West, don't know if you're interested in this PR for AVRO-2003 as well (allowing users to prevent accidental data loss, due to schema typing issues)

          Show
          epkanol Anders Sundelin added a comment - Ping Elliot West , don't know if you're interested in this PR for AVRO-2003 as well (allowing users to prevent accidental data loss, due to schema typing issues)

            People

            • Assignee:
              epkanol Anders Sundelin
              Reporter:
              epkanol Anders Sundelin
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:

                Development