[KYLIN-2365] multi FactDistinctColumnsReducer task write the same output file

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: v1.6.0
• Component/s: NRT Streaming
• Assignee: Unassigned
• Reporter: zheng zhifeng
• Created: 08/Jan/17 07:01
• Updated: 09/Aug/17 05:06
• Resolved: 17/Jan/17 05:50

Description

Multi FactDistinctColumnsReducer task will write the
distinct columns to the same output file if the mapreduce
property mapreduce.reduce.speculative is true. This will
make the mapreduce job failed or the output file is redundancy.
I think it shoud set mapreduce.reduce.speculative to false on
Extract Fact Table Distinct Columns step.

Issue Links

• is duplicated by: KYLIN-2242 Directly write hdfs file in reducer is dangerous