[ARROW-8860] [C++] IPC/Feather decompression broken for nested arrays

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 1.0.0
• Component/s: C++
• Labels: pull-request-available
• External issue URL: https://github.com/apache/arrow/issues/24999
• Assignee: Joris Van den Bossche
• Reporter: Joris Van den Bossche
• Created: 19/May/20 18:38
• Updated: 11/Jan/23 11:03
• Resolved: 11/Jun/20 13:14

Description

When writing a table with a Struct typed column, this is read back with garbage values when using compression (which is the default):

>>>  table = pa.table({'col': pa.StructArray.from_arrays([[0, 1, 2], [1, 2, 3]], names=["f1", "f2"])})

# roundtrip through feather
>>> feather.write_feather(table, "test_struct.feather")
>>> table2 = feather.read_table("test_struct.feather")

>>> table2.column("col")
<pyarrow.lib.ChunkedArray object at 0x7f0b0c4d7728>
[
  -- is_valid: all not null
  -- child 0 type: int64
    [
      24,
      1261641627085906436,
      1369095386551025664
    ]
  -- child 1 type: int64
    [
      24,
      1405756815161762308,
      281479842103296
    ]
]

When not using compression, it is read back correctly:

>>> feather.write_feather(table, "test_struct.feather", compression="uncompressed")                                                                                                                           
>>> table2 = feather.read_table("test_struct.feather")                                                                                                                                                        

>>> table2.column("col")                                                                                                                                                                                      
<pyarrow.lib.ChunkedArray object at 0x7f0b0e466778>
[
  -- is_valid: all not null
  -- child 0 type: int64
    [
      0,
      1,
      2
    ]
  -- child 1 type: int64
    [
      1,
      2,
      3
    ]
]

Issue Links

• duplicates: ARROW-8868 [Python] Feather format cannot store/retrieve lists correctly?
• links to: GitHub Pull Request #7233