Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-2366

multi FactDistinctColumnsReducer task write the same output file

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: v1.6.0
    • Fix Version/s: None
    • Component/s: NRT Streaming
    • Labels:
      None

      Description

      Multi FactDistinctColumnsReducer task will write the
      distinct column to the same output file if the mapreduce
      property mapreduce.reduce.speculative is true. This will
      make the mapreduce job failed or the output file is redundancy.
      I think it shoud set mapreduce.reduce.speculative to false on
      Extract Fact Table Distinct Columns step.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                zhengzfand zheng zhifeng
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: