[ARROW-13126] Read out only the required columns from a Feather file on Disk

• Type: New Feature
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Component/s: C++
• External issue URL: https://github.com/apache/arrow/issues/28827
• Assignee: Unassigned
• Reporter: Jayjeet Chakraborty
• Created: 19/Jun/21 13:12
• Updated: 11/Jan/23 08:31
• Resolved: 21/Jun/21 12:33

Description

The LZ4 compressed feather format works really well and is quite nice and fast than Parquet when reading all the columns out. For single columns, looks like feather format does not yet support the capability to read out only the required columns from the disk. Are there any plans to add support for this? Here are some numbers to support my claim from an experiment with a table with 17 columns and 5 million rows in uncompressed parquet and LZ4 compressed feather format. No memory mapping involved.

 
pq_all_cols: 0.4179724836349487 ms
feather_all_cols: 0.26202451705932617 ms
pq_single_col: 0.10951032638549804 ms
feather_single_col: 0.2119576358795166 ms

Issue Links

• relates to: ARROW-12683 [C++] Enable fine-grained I/O (coalescing) in IPC reader