[HUDI-2762] Ensure hive can query insert only logs in MOR

• Type: Improvement
• Status: Reopened
• Priority: Critical
• Resolution: Unresolved
• Component/s: hive, reader-core
• Story Points: 2
• Epic Link: Streaming Read
• Assignee: Alexey Kudinkin
• Reporter: Rajesh Mahindra

Reviewers

Shiyan Xu

• Created: 15/Nov/21 17:10
• Updated: 10/Mar/23 01:50

Description

Currently, we are able to query MOR tables that have base parquet files with inserts an logs files with updates. However, we are currently unable to query tables with insert only log files. Both _ro and _rt tables are returning 0 rows. However, hms does create the table and partitions for the table. 

 

One sample table is here:

https://s3.console.aws.amazon.com/s3/buckets/debug-hive-site?prefix=database/&region=us-east-2

 

 

Issue Links

• is blocked by: HUDI-3081 Revisiting Read Path Infra across Query Engines
• is duplicated by: HUDI-3289 Fix the read path to read records from only log files in MOR table
• Testing discovered: HUDI-3348 HoodieRealtimeFileSplit losing info when serialized/deserialized