Uploaded image for project: 'ORC'
  1. ORC
  2. ORC-255

Add support for reading ACID2 files to ORC

    XMLWordPrintableJSON

    Details

    • Type: New Feature
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: ACID
    • Labels:
      None

      Description

      When ORC was split out of Hive the reading and writing of ACID files was left in Hive. This blocks non-Hive users from reading or writing ACID. I propose to add support for ACID to ORC.

      At this point I only propose to add support for ACID 2 (that is, the version that will be released in Hive 3, which simplifies the storage to have only inserts and deletes (updates are an insert plus a delete)). Also, I am initially at least just adding support via the vector batch interface, not the row by row interface.

      Also, note that to use this readers and writers would still have to interact with the Hive metastore to get a list of valid transactions and acquire appropriate locks.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gates Alan Gates
                Reporter:
                gates Alan Gates
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: