Uploaded image for project: 'Apache Avro'
  1. Apache Avro
  2. AVRO-3218

Pass LogicalType to BytesDecimalSchema in Python

VotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.10.2
    • 1.11.1
    • python

    Description

      I get an error when parsing a schema which contains a field with `logicalType=decimal` in Python. Spesifically, I am using avro.schema.parse() on a schema which have a field on the following format: 

      { "name": "myField", "type": [ "null", { "type": "bytes", "logicalType": "decimal", "precision": 12, "scale": 2 } ]}
      

      Eventually, make_bytes_decimal_schema() is called. At this point, the variable other_props looks like this (and is fine):

       

      other_props = {'logicalType': 'decimal', 'precision': 12, 'scale': 2}

       

      Unlike the other logical types, the variable other_props is not passed on to BytesDecimalSchema. The result is that the resulting logical_schema is missing the key-value pair 'logicalType': 'decimal'. This blocks me from writing to a kafka topic, because the schema is parsed incorrectly.  The resulting field looks like this:

      {"type": ["null", {"type": "bytes", "precision": 12, "scale": 2}], "name": "myField"}
      

       

      The suggested fix is to simply pass other_props as an argument into BytesDecimalSchema(). The attached file issue.pycontains code to reproduce the bug

       

      This commit contains the suggested changes: AVRO-3218 Fix parsing of logicalType = decimal · vegraux/avro@b5c4a78 (github.com)

      Attachments

        1. issue.py
          0.4 kB
          Vegard Solberg

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            vegardso Vegard Solberg
            vegardso Vegard Solberg
            Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 2h 10m
                2h 10m

                Slack

                  Issue deployment