[HIVE-26146] Handle missing hive.acid.key.index in the fixacidkeyindex utility

• Type: Improvement
• Status: Closed
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 4.0.0-alpha-2
• Fix Version/s: 4.0.0-alpha-2
• Component/s: ORC, Transactions
• Labels: pull-request-available
• Assignee: Alessandro Solimando
• Reporter: Alessandro Solimando
• Created: 16/Apr/22 08:11
• Updated: 16/Nov/22 13:50
• Resolved: 19/Apr/22 15:13

Description

There is a utility in hive which can validate/fix corrupted hive.acid.key.index:

hive --service fixacidkeyindex $orcfilepath

At the moment the utility throws a NPE if the hive.acid.key.index metadata entry is missing:

ERROR checking /hive-dev-box/multistripe_ko_acid.orc
java.lang.NullPointerException
        at org.apache.hadoop.hive.ql.io.orc.FixAcidKeyIndex.validate(FixAcidKeyIndex.java:183)
        at org.apache.hadoop.hive.ql.io.orc.FixAcidKeyIndex.checkFile(FixAcidKeyIndex.java:147)
        at org.apache.hadoop.hive.ql.io.orc.FixAcidKeyIndex.checkFiles(FixAcidKeyIndex.java:130)
        at org.apache.hadoop.hive.ql.io.orc.FixAcidKeyIndex.main(FixAcidKeyIndex.java:106)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.apache.hadoop.util.RunJar.run(RunJar.java:308)
        at org.apache.hadoop.util.RunJar.main(RunJar.java:222)

The aim of this ticket is to handle such case in order to support re-generating this metadata entry even when it is missing.

Issue Links

• relates to: HIVE-26147 OrcRawRecordMerger throws NPE when hive.acid.key.index is missing for an acid file
• relates to: HIVE-24475 Generalize fixacidkeyindex utility
• links to: GitHub Pull Request #3216