Avro
  1. Avro
  2. AVRO-1518

Python client support decimal.Decimal types -> double encoding / decoding

    Details

    • Type: Improvement Improvement
    • Status: Patch Available
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 1.7.8
    • Component/s: None
    • Labels:
      None
    • Tags:
      python

      Description

      Python standard library > 2.4 provides a Decimal type that has much better semantics then standard binary float. Avro library should be able to accept Decimal's and encode them as doubles.
      (https://docs.python.org/2/library/decimal.html)

      I also believe it should, by default, turn Avro double's into Decimal object instead of a float.

      Simple patch allows for encoding a Decimal into an Avro double

      --- io.py	2014-05-23 13:41:14.000000000 -0700
      +++ /Users/sreynolds/Projects/avro-1.7.6 2/src/avro/io.py	2014-05-23 13:44:03.000000000 -0700
      @@ -46,6 +46,11 @@ try:
       except ImportError:
       	import simplejson as json
      
      +try:
      +        from decimal import Decimal
      +except ImportError:
      +        Decimal = float
      +
       #
       # Constants
       #
      @@ -117,7 +122,7 @@ def validate(expected_schema, datum):
                   and LONG_MIN_VALUE <= datum <= LONG_MAX_VALUE)
         elif schema_type in ['float', 'double']:
           return (isinstance(datum, int) or isinstance(datum, long)
      -            or isinstance(datum, float))
      +            or isinstance(datum, float) or instance(datum, Decimal))
         elif schema_type == 'fixed':
           return isinstance(datum, str) and len(datum) == expected_schema.size
         elif schema_type == 'enum':
      

        Activity

        Hide
        Steven Moy added a comment -

        Encoding is doable. For decoding, is that going to be a runtime option during decoding by expanding API?

        Also, the patch will need to add an unit test to exercise the encode decimal.Decimal in the existing test suite.

        Show
        Steven Moy added a comment - Encoding is doable. For decoding, is that going to be a runtime option during decoding by expanding API? Also, the patch will need to add an unit test to exercise the encode decimal.Decimal in the existing test suite.
        Hide
        Doug Cutting added a comment -

        Looks reasonable to me. Any Pythonistas have opinions about this?

        Show
        Doug Cutting added a comment - Looks reasonable to me. Any Pythonistas have opinions about this?

          People

          • Assignee:
            Scott Reynolds
            Reporter:
            Scott Reynolds
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development