[ORC-1401] [C++] allow writing an intermediate footer

• Type: Improvement
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 1.9.0
• Component/s: C++
• Labels: feature pull-request-available

Language

C++

• Assignee: Hinko Kocevar
• Reporter: Hinko Kocevar
• Created: 29/Mar/23 19:56
• Updated: 27/Nov/23 21:26
• Resolved: 28/Jun/23 22:03

Description

Current C++ code base does not support writing intermediate footers as Java code base does.

It would be beneficial to the C++ based writers to have an ability to write intermediate footer(s) to the opened file while streaming data into it. The benefit is not losing the so far persisted data (after flushing stripes) in case of writer failure in closing the file (application crash).

The writer that wants to support the feature would follow similar approach as Hive does in writing a side file with the latest footer offsets. The reader would be able to determine the location of the last valid footer by using the last footer location from the side file (instead of assuming that the footer is at the end of the file).

I'll open a Github Draft PR that will reference this issue.

Issue Links

• links to: GitHub Pull Request #1455