[KYLIN-1861] Hive may fail to create flat table with "GC overhead error"

• Type: Test
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: v1.5.3
• Assignee: Shao Feng Shi
• Reporter: Shao Feng Shi
• Created: 08/Jul/16 10:42
• Updated: 28/Jul/16 06:48
• Resolved: 11/Jul/16 02:53

Description

in conf/kylin_hive_conf.xml, it enables "hive.auto.convert.join.noconditionaltask", which is the optimization about converting common join into mapjoin based on the input file size, and give a big value (300000000, 300MB) to "hive.auto.convert.join.noconditionaltask.size". So if the memory of the mapper isn't enough, it will cause the error.

To fix the error, user can reduce the "hive.auto.convert.join.noconditionaltask.size" in conf/kylin_hive_conf.xml, and then retry the job.