[KYLIN-1566] Add and use a separate kylin_job_conf.xml for in-mem cubing

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v1.5.0
• Fix Version/s: v1.5.2
• Component/s: Job Engine
• Assignee: Shao Feng Shi
• Reporter: Shao Feng Shi
• Created: 07/Apr/16 03:45
• Updated: 26/May/16 09:16
• Resolved: 11/May/16 06:36

Description

From v1.5.0, kylin has two cubing algorithm: by-layer and by-split(in-mem).

The in-mem cubing require more memory resource for the mapper; so far it is configured by parameter "kylin.job.cubing.inmem.mrjob_conf_override" with default value "mapreduce.map.java.opts=-Xmx2700m; mapreduce.map.memory.mb=3072; mapreduce.task.io.sort.mb=200"; This is a little confusing as user can also configure such parameters in kylin_job_conf.xml

A better way is adding a new config file for in-mem cubing job, moving the in-mem specific mapreduce parameters there.