Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6954

Disable erasure coding for files that are uploaded to the MR staging area

          Details

          • Type: Improvement
          • Status: Resolved
          • Priority: Major
          • Resolution: Fixed
          • Affects Version/s: 3.0.0-alpha4
          • Fix Version/s: 3.0.0-beta1
          • Component/s: client
          • Labels:
            None
          • Hadoop Flags:
            Reviewed

            Description

            Depending on the encoder/decoder used and the type or MR workload, EC might negatively affect the performance of an MR job if too many files are localized.

            In such a scenario, users might want to disable EC in the staging area to speed up the execution.

              Attachments

              1. MAPREDUCE-6954-001.patch
                4 kB
                Peter Bacsko
              2. MAPREDUCE-6954-002.patch
                8 kB
                Peter Bacsko
              3. MAPREDUCE-6954-003.patch
                8 kB
                Peter Bacsko
              4. MAPREDUCE-6954-004.patch
                8 kB
                Peter Bacsko

                Issue Links

                breaks

                Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-7059 Downward Compatibility issue: MR job fails because of unknown setErasureCodingPolicy method from 3.x client to HDFS 2.x cluster

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved

                Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6968 Staging directory erasure coding config property has a typo

                • Major - Major loss of function.
                • Resolved
                relates to

                Sub-task - The sub-task of the issue HDFS-12919 RBF: Support erasure coding methods in RouterRpcServer

                • Critical - Crashes, loss of data, severe memory leak.
                • Resolved

                  Activity

                    People

                    • Assignee:
                      pbacsko Peter Bacsko
                      Reporter:
                      pbacsko Peter Bacsko
                    • Votes:
                      0 Vote for this issue
                      Watchers:
                      5 Start watching this issue

                      Dates

                      • Created:
                        Updated:
                        Resolved: