Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2162

[Python/C++] Decimal Values with too-high precision are multiplied by 100

    Details

      Description

      From GitHub:

      This works as expected:

      >>> pyarrow.array([decimal.Decimal('1.23')], pyarrow.decimal128(10,2))[0]
      Decimal('1.23')
      

      Storing an extra digit of precision multiplies the stored value by a factor of 100:

      >>> pyarrow.array([decimal.Decimal('1.234')], pyarrow.decimal128(10,2))[0]
      Decimal('123.40')
      

      Ideally I would get an exception since the value I'm trying to store doesn't fit in the declared type of the array. It would be less good, but still ok, if the stored value were 1.23 (truncating the extra digit). I didn't expect pyarrow to silently store a value that differs from the original value by a factor of 100.

      I originally thought that the code was incorrectly multiplying through by an extra factor of 10**scale, but that doesn't seem to be the case. If I change the scale, it always seems to be a factor of 100

      >>> pyarrow.array([decimal.Decimal('1.2345')], pyarrow.decimal128(10,3))[0]
      Decimal('123.450')
      I see the same behavior if I use floating point to initialize the array rather than Python's decimal type.
      

      I searched for open github and JIRA for open issues but didn't find anything related to this. I am using pyarrow 0.8.0 on OS X 10.12.6 using python 2.7.14 installed via Homebrew

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                cpcloud Phillip Cloud
                Reporter:
                cpcloud Phillip Cloud
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: