[HIVE-11983] Hive streaming API uses incorrect logic to assign buckets to incoming records

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.2.1
• Fix Version/s: 1.3.0, 2.0.0
• Component/s: HCatalog, Transactions
• Labels: streaming streaming_api
• Assignee: Roshan Naik
• Reporter: Roshan Naik
• Created: 28/Sep/15 21:49
• Updated: 08/Apr/16 18:49
• Resolved: 05/Oct/15 18:55

Description

The Streaming API tries to distribute records evenly into buckets.
All records in every Transaction that is part of TransactionBatch goes to the same bucket and a new bucket number is chose for each TransactionBatch.

Fix: API needs to hash each record to determine which bucket it belongs to.

Issue Links

• blocks: HIVE-11914 When transactions gets a heartbeat, it doesn't update the lock heartbeat.
• blocks: HIVE-12025 refactor bucketId generating code
• is related to: HIVE-11672 Hive Streaming API handles bucketing incorrectly
• is related to: HIVE-13403 Make Streaming API not create empty buckets