[ORC-562] Don't wrap readerSchema in acidSchema, if readerSchema is already acid

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.6, 1.6.0
• Fix Version/s: 1.5.8, 1.6.2, 1.7.0
• Component/s: Java
• Assignee: László Pintér
• Reporter: László Pintér
• Created: 24/Oct/19 08:05
• Updated: 25/Nov/19 16:16
• Resolved: 05/Nov/19 18:26

Description

create table tbl1 (a int, b string) partitioned by (ds string) stored as orc tblproperties ('transactional'='true');
insert into tbl1 partition (ds) values (1, 'fred', 'today'), (2, 'wilma', 'yesterday');

As this table is transactional, all the modifications will generate a new delta directory, containing a delta file in orc format. The schema of this file will be

struct<operation:int,originaltransaction:bigint,bucket:int,rowid:bigint,currenttransaction:bigint,row:struct<a:int,b:string>>

If I create a new partitioned table with the very same schema, and change the partition location to one of the delta directories, I would assume that I would be able to run queries against the contents of the delta file.

Right now this is not possible in orc, because the original readerschema is wrapped in acidschema again, regardless that the readerschema is already acid.

struct<operation:int,originalTransaction:bigint,bucket:int,rowId:bigint,currentTransaction:bigint,row:struct<operation:int,originaltransaction:bigint,bucket:int,rowid:bigint,currenttransaction:bigint,row:struct<a:int,b:string>>>

Issue Links

• blocks: HIVE-20934 ACID: Query based compactor for minor compaction
• links to: GitHub Pull Request #442