[ARROW-377] Python: Add support for conversion of Pandas.Categorical

• Type: New Feature
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.2.0
• Component/s: Python
• Labels: newbie
• External issue URL: https://github.com/apache/arrow/issues/15950
• Assignee: Wes McKinney
• Reporter: Uwe Korn
• Created: 15/Nov/16 21:01
• Updated: 11/Jan/23 07:08
• Resolved: 24/Jan/17 00:28

Description

At the moment conversion from pandas.Categorical columns fails with ArrowException: Invalid: only handle 1-dimensional arrays. As a better alternative, we should provide one of the following solutions:

• Convert the categorical column to a string (Pandas type object) column, then use the conversion routines for strings. Add some metadata to the Arrow column that it was initially a Pandas string column so that in the case of a roundtrip, it will be a categorical column again.
• Implement the conversion of the column to a dictionary-encoded Arrow column. This is the preferred solution but may be more complicated to implement as certain requirements have not yet been implemented.