[HIVE-3593] Output files of SMB join grow indefinitely

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 0.10.0
• Component/s: Query Processor
• Assignee: Kevin Wilfong
• Reporter: Kevin Wilfong
• Created: 17/Oct/12 21:39
• Updated: 17/Oct/12 23:07
• Resolved: 17/Oct/12 23:07

Description

The output files of a SMB join are prefixed by the big table's partition spec that was used to create them. The length of the bucket number portion of the file name is updated to be the same length as the length of the task ID. Since the task ID is the name of the file, this means that if the output of a SMB join is used as the big table of another SMB join, the output files will increase by the size of the original partition spec. Compound this and the file size can grow indefinitely.

Issue Links

• is broken by: HIVE-3536 Output of sort merge join is no longer bucketed