[GOBBLIN-89] Hive compaction Delta structure

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Labels: question
• External issue URL: https://github.com/linkedin/gobblin/issues/1179
• Assignee: Unassigned
• Reporter: Abhishek Tiwari
• Created: 19/Jul/17 07:40
• Updated: 07/Aug/17 17:53

Description

i was trying to run Hive Compaction for Entity Snapshot with one delta table. But i have realized when consuming delta it doesn't merge the multiple versions of same entity in single delta table.

Is this a condition, that only one entity version should be present in one delta table. Or i am missing something?
For ex: i have delta of Employee in one table
id name salary
1 x a
1 x b
1 x.y b

in final output after compaction i am getting all the records with id 1.

Github Url : https://github.com/linkedin/gobblin/issues/1179
Github Reporter : chandanbansal
Github Created At : 2016-08-04T15:28:15Z
Github Updated At : 2017-01-12T05:00:45Z