Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-18662

hive.acid.key.index is missing entries

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0
    • Component/s: Transactions
    • Labels:
      None
    • Target Version/s:

      Description

      OrcRecordUpdater.KeyIndexBuilder stores an index in ORC footer where each entry is the last ROW__ID of each stripe.  In acid1 this is used to filter the events from delta file when merging with part of the base.

       

      as can be seen in TestTxnCommands.testVersioning() (added in HIVE-18659) the hive.acid.key.index is empty.  

       

      This is because very little data is written and WriterImpl.flushStripe() is not called except when {

      {WriterImpl.close()}

      is called.  In the later, WriterCallback.preFooterWrite() is called before preStripeWrite and so KeyIndexBuilder.preFooterWriter() records nothing in hive.acid.key.index

       

      need to investigate if this is an issue, in particular acid 2

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                ekoifman Eugene Koifman
                Reporter:
                ekoifman Eugene Koifman
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: