Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-1401

[C++] allow writing an intermediate footer

    XMLWordPrintableJSON

Details

    Description

      Current C++ code base does not support writing intermediate footers as Java code base does.

      It would be beneficial to the C++ based writers to have an ability to write intermediate footer(s) to the opened file while streaming data into it. The benefit is not losing the so far persisted data (after flushing stripes) in case of writer failure in closing the file (application crash).

      The writer that wants to support the feature would follow similar approach as Hive does in writing a side file with the latest footer offsets. The reader would be able to determine the location of the last valid footer by using the last footer location from the side file (instead of assuming that the footer is at the end of the file).

      I'll open a Github Draft PR that will reference this issue.

      Attachments

        Issue Links

          Activity

            People

              hinxx Hinko Kocevar
              hinxx Hinko Kocevar
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: