Uploaded image for project: 'Apache Arrow'
  1. Apache Arrow
  2. ARROW-2228

[Python] Unsigned int type for arrow Table not supported

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: Python
    • Labels:
      None
    • Environment:
      Ubuntu 16.04
      python3.6.3

      Description

      Running this python one-liner

       

      // code pa.Table.from_pandas(pd.DataFrame({'foo': [np.array([10000000000000000000], dtype=np.uint64)]}))
      

      I get

      // code ---------------------------------------------------------------------------
      ArrowInvalid                              Traceback (most recent call last)
      <ipython-input-115-cbc9670d125f> in <module>()
      ----> 1 pa.Table.from_pandas(pd.DataFrame({'foo': [np.array([10000000000000000000], dtype=np.uint64)]}))
      
      ~/.virtualenvs/log-archive/lib/python3.6/site-packages/pyarrow/table.pxi in pyarrow.lib.Table.from_pandas (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:44927)()
      
      ~/.virtualenvs/log-archive/lib/python3.6/site-packages/pyarrow/pandas_compat.py in dataframe_to_arrays(df, schema, preserve_index, nthreads)
          348         arrays = [convert_column(c, t)
          349                   for c, t in zip(columns_to_convert,
      --> 350                                   convert_types)]
          351     else:
          352         from concurrent import futures
      
      ~/.virtualenvs/log-archive/lib/python3.6/site-packages/pyarrow/pandas_compat.py in <listcomp>(.0)
          347     if nthreads == 1:
          348         arrays = [convert_column(c, t)
      --> 349                   for c, t in zip(columns_to_convert,
          350                                   convert_types)]
          351     else:
      
      ~/.virtualenvs/log-archive/lib/python3.6/site-packages/pyarrow/pandas_compat.py in convert_column(col, ty)
          343
          344     def convert_column(col, ty):
      --> 345         return pa.array(col, from_pandas=True, type=ty)
          346
          347     if nthreads == 1:
      
      ~/.virtualenvs/log-archive/lib/python3.6/site-packages/pyarrow/array.pxi in pyarrow.lib.array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:29224)()
      
      ~/.virtualenvs/log-archive/lib/python3.6/site-packages/pyarrow/array.pxi in pyarrow.lib._ndarray_to_array (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:28465)()
      
      ~/.virtualenvs/log-archive/lib/python3.6/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status (/arrow/python/build/temp.linux-x86_64-3.6/lib.cxx:8270)()
      
      ArrowInvalid: trying to convert NumPy type int64 but got uint64
      

       

      the problem possibly relies on the fact that from_pandas doesn't handle the conversion from python object to unsigned integer.

       

        Attachments

          Activity

            People

            • Assignee:
              wesmckinn Wes McKinney
              Reporter:
              mdalp Marcello
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: