[KYLIN-1385] Enhance FactDistinctColumnsJob to use multiple reducers

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: v1.2, v1.4.0
• Fix Version/s: v1.5.0
• Component/s: Job Engine
• Assignee: Shao Feng Shi
• Reporter: Shao Feng Shi
• Created: 31/Jan/16 08:06
• Updated: 23/Mar/16 13:31
• Resolved: 23/Mar/16 13:31

Description

So far the FactDistinctColumnsJob is using 1 reducer to merge the distinct values for all dimension columns on fact table. When there are 1 or multiple ultra high cardinality columns, it couldn't gain concurrency; It should dispatch the workload to multiple reducers.