[ARROW-16580] [Python] Multiindex levels order is not preserved after a from_pandas/to_pandas

• Type: Bug
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: 8.0.0
• Component/s: Python
• Labels: pandas
• External issue URL: https://github.com/apache/arrow/issues/31937
• Assignee: Unassigned
• Reporter: Fabien Aulaire
• Created: 14/May/22 21:53
• Updated: 11/Jan/23 11:45

Description

Hello,

Not sure if it's the good place to report this issue but here is what I saw when I tried to convert a multi indexed dataframe (columns) to Table and convert it back to pandas:

 

import pandas as pd
import pyarrow

# pandas version '1.4.1'
# pyarrow version '8.0.0'

df = pd.DataFrame([[100,300, 400], [200,500, 600]], columns=['Toyota', 'Ford', 'Alfa'])
concatenated = pd.concat([df, df*2], axis=1, keys=['foo', 'bar'])
concatenated.columns.names = ['l1', 'l2']

table = pyarrow.Table.from_pandas(concatenated)
from_table_df = table.to_pandas()

from_table_df.columns.levels # == FrozenList([['bar', 'foo'], ['Alfa', 'Ford', 'Toyota']])
concatenated.columns.levels # == FrozenList([['foo', 'bar'], ['Toyota', 'Ford', 'Alfa']])

the order of columns levels is not preserved.