[ARROW-15340] [C++] Try and fetch IPC footer in one read instead of two

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: C++
• Labels: good-second-issue
• External issue URL: https://github.com/apache/arrow/issues/30827
• Assignee: Unassigned
• Reporter: Weston Pace
• Created: 14/Jan/22 21:08
• Updated: 11/Jan/23 11:36

Description

To concisely read the IPC footer we must do two reads. First read the footer length and then read the footer contents.

On high latency filesystems we can cut down some of this startup time by assuming the footer is going to be relatively small and reading the last ~1MB of the file. Then we only need to do two reads if the footer is larger.

This means most of the time we can usually get away with 1 read at the expense of possibly reading more data than needed.