Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-14995

Configuration should be forced to load resources as an option to avoid loss of configuration results lead to data loss

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Critical
    • Resolution: Unresolved
    • None
    • None
    • conf
    • None

    Description

      MapReduce Job, the task initialization by job.xml , but if missing the soft link of "job.xml" file,will do nothing under the quietmode of Configuration.
      Such as map tasks as directly output to HDFS without reduce tasks of job, if map task missing job.xml, it's will run with default behavior, just as IdentityMapper and output to local, and resulting in data loss.

      Attachments

        1. HADOOP-14995-trunk-001.patch
          4 kB
          DENG FEI
        2. log.png
          579 kB
          DENG FEI
        3. data_loss.png
          276 kB
          DENG FEI
        4. job_detail.png
          256 kB
          DENG FEI
        5. normal_task.png
          234 kB
          DENG FEI

        Issue Links

          is required by

          Improvement - An improvement or enhancement to an existing feature or task. MAPREDUCE-7009 Ensure Job Configuration File is loadable for Task

          • Critical - Crashes, loss of data, severe memory leak.
          • Open

          Activity

            People

              Unassigned Unassigned
              Deng FEI DENG FEI
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated: