[AMBARI-5455] Ambari configuration for map join conversion and tez container size seems wrong

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: 1.5.1
• Fix Version/s: 1.5.1
• Component/s: ambari-web
• Assignee: Srimanth Gunturi
• Reporter: Srimanth Gunturi
• Created: 14/Apr/14 05:03
• Updated: 14/Apr/14 19:35
• Resolved: 14/Apr/14 07:21

Description

For hive:

hive.auto.convert.join.noconditionaltask.size is set to 1000000000

• This should be a fraction (1/3) of the container size.

hive.tez.java.opts has "-Xmx1024m"

This is different from both map and reduce sizes. Desired values are:

• map size if map size > 2g else reduce size

map size is set on the same cluster to ~500mb

The settings as the are will lead to many failed queries because the mapjoin conversion is to aggressive. If we don't change the container sizes based on cluster configs we will see wide spread problems with containers being killed or perf problems.