[ARROW-4370] [Python] Table to pandas conversion fails for list of bool

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.12.0
• Fix Version/s: 0.13.0
• Component/s: Python
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/20937
• Assignee: Uwe Korn
• Reporter: Francisco Sanchez
• Created: 25/Jan/19 11:40
• Updated: 11/Jan/23 07:33
• Resolved: 09/Feb/19 21:03

Description

When converting a Table with a list of bool to pandas it doesn't work and throws an error. For list of int is working correctly.

 In [94]: x1 = [1,2,3,4]
 In [95]: f1 = pa.field('test', pa.list_(pa.int32()))
 In [96]: array1 = pa.array([x1], type=f1.type)
 In [97]: pa.Table.from_arrays([array1], names=['col1']).to_pandas()
 Out[97]: 
 col1
 0 [1, 2, 3, 4]
 In [98]: x2 = [True, True, False, False]
 In [99]: f2 = pa.field('test', pa.list_(pa.bool_()))
 In [100]: array2 = pa.array([x2], type=f2.type)
 In [101]: pa.Table.from_arrays([array2], names=['col1']).to_pandas()
 ---------------------------------------------------------------------------
 ArrowNotImplementedError Traceback (most recent call last)
 <ipython-input-101-04ab2e0d4c8f> in <module>()
 ----> 1 pa.Table.from_arrays([array2], names=['col1']).to_pandas()
 ~/.virtualenvs/py3-gpu/lib/python3.5/site-packages/pyarrow/array.pxi in pyarrow.lib._PandasConvertible.to_pandas()
 ~/.virtualenvs/py3-gpu/lib/python3.5/site-packages/pyarrow/table.pxi in pyarrow.lib.Table._to_pandas()
 ~/.virtualenvs/py3-gpu/lib/python3.5/site-packages/pyarrow/pandas_compat.py in table_to_blockmanager(options, table, categories, ignore_metadata)
 630 
 631 blocks = _table_to_blocks(options, block_table, pa.default_memory_pool(),
 --> 632 categories)
 633 
 634 # Construct the row index
 ~/.virtualenvs/py3-gpu/lib/python3.5/site-packages/pyarrow/pandas_compat.py in _table_to_blocks(options, block_table, memory_pool, categories)
 803 # Convert an arrow table to Block from the internal pandas API
 804 result = pa.lib.table_to_blocks(options, block_table, memory_pool,
 --> 805 categories)
 806 
 807 # Defined above
 ~/.virtualenvs/py3-gpu/lib/python3.5/site-packages/pyarrow/table.pxi in pyarrow.lib.table_to_blocks()
 ~/.virtualenvs/py3-gpu/lib/python3.5/site-packages/pyarrow/error.pxi in pyarrow.lib.check_status()
 ArrowNotImplementedError: Not implemented type for list in DataFrameBlock: bool

Issue Links

• links to: GitHub Pull Request #3593