[KYLIN-4433] When uhc step is turned on, Build Dimension Dictionary job cannot get correct configuration

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v2.6.4
• Fix Version/s: v3.0.2, v2.6.6
• Component/s: Job Engine
• Assignee: ChangjiGuo
• Reporter: ChangjiGuo
• Created: 25/Mar/20 11:36
• Updated: 26/Mar/20 06:52
• Resolved: 26/Mar/20 06:52

Description

Hi guys :

    In the Configuration overrides step of Cube Designer, I added the following configuration:

kylin.engine.mr.build-uhc-dict-in-additional-step = true

    When building the cube, it will take an extra step to handle the UHC dimensions, but I found that in the step of 'Build Dimension Dictionary', it still builds the dictionary that has been built in the previous step which takes double the time to build the dictionary.

    When I traced the source code, I found that our expected code was not executed due to config.isBuildUHCDictWithMREnabled() == false :

if (config.isBuildUHCDictWithMREnabled() && uhcColumns.contains(col)) {
    colDir = new Path(dictPath, col.getIdentity());
} else {
    colDir = new Path(factColumnsInputPath, col.getIdentity());
}

    The root cause is that the configuration used in this step is the environment configuration, but our parameters are set in the cube:

final KylinConfig config = KylinConfig.getInstanceFromEnv();

    I think the configuration of the cube should be used here instead of the environment configuration.

Issue Links

• links to: GitHub Pull Request #1168