[ARROW-2017] Array initialization with large (>2**31-1) uint64 values fails

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.8.0
• Fix Version/s: 0.9.0
• Component/s: Python
• Labels: pull-request-available
• Environment: python 3.6, 3.5, 2.7, on OSX and Linux
• Assignee: Wes McKinney
• Reporter: Ian Roddis
• Created: 23/Jan/18 01:58
• Updated: 25/Jan/18 22:40
• Resolved: 25/Jan/18 22:40

Description

Trying to create pyarrow arrays with large values causes a 'too big to convert' error:

Python 3.5.1 (default, Jun  6 2016, 14:30:01)
Type "copyright", "credits" or "license" for more information.

IPython 5.1.0 – An enhanced Interactive Python.
?         -> Introduction and overview of IPython's features.
%quickref -> Quick reference.
help      -> Python's own help system.
object?   -> Details about 'object', use 'object??' for extra details.

In [1]: import pyarrow as pa

In [2]: pa.array([2**63-1], type=pa.uint64())
Out[2]:
<pyarrow.lib.UInt64Array object at 0x1050e73b8>
[
  9223372036854775807
]

In [3]: pa.array([2**63], type=pa.uint64())
---------------------------------------------------------------------------
ArrowException                            Traceback (most recent call last)
<ipython-input-3-0d103b16c512> in <module>()
----> 1 pa.array([2**63], type=pa.uint64())

/Users/roddis/.pyenv/versions/3.5.1/envs/g3.5.1/lib/python3.5/site-packages/pyarrow/array.pxi in pyarrow.lib.array (/Users/travis/build/BryanCutler/arrow-dist/arrow/python/build/temp.macosx-10.6-intel-3.5/lib.cxx:29283)()

/Users/roddis/.pyenv/versions/3.5.1/envs/g3.5.1/lib/python3.5/site-packages/pyarrow/array.pxi in pyarrow.lib._sequence_to_array (/Users/travis/build/BryanCutler/arrow-dist/arrow/python/build/temp.macosx-10.6-intel-3.5/lib.cxx:27945)()

/Users/roddis/.pyenv/versions/3.5.1/envs/g3.5.1/lib/python3.5/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status (/Users/travis/build/BryanCutler/arrow-dist/arrow/python/build/temp.macosx-10.6-intel-3.5/lib.cxx:9068)()

ArrowException: Unknown error: int too big to conver

Issue Links

• links to: GitHub Pull Request #1504