Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-3150

Move task file promotion into the task

    Details

    • Type: New Feature
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.19.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Hide
      Moved task file promotion to the Task. When the task has finished, it will do a commit and is declared SUCCEDED. Job cleanup is done by a separate task. Job is declared SUCCEDED/FAILED after the cleanup task has finished. Added public classes org.apache.hadoop.mapred.JobContext, TaskAttemptContext, OutputCommitter and FileOutputCommiitter. Added public APIs: public OutputCommitter getOutputCommitter() and
      public void setOutputCommitter(Class<? extends OutputCommitter> theClass) in org.apache.hadoop.mapred.JobConf
      Show
      Moved task file promotion to the Task. When the task has finished, it will do a commit and is declared SUCCEDED. Job cleanup is done by a separate task. Job is declared SUCCEDED/FAILED after the cleanup task has finished. Added public classes org.apache.hadoop.mapred.JobContext, TaskAttemptContext, OutputCommitter and FileOutputCommiitter. Added public APIs: public OutputCommitter getOutputCommitter() and public void setOutputCommitter(Class<? extends OutputCommitter> theClass) in org.apache.hadoop.mapred.JobConf

      Description

      We need to move the task file promotion from the JobTracker to the Task and move it down into the output format.

        Attachments

        1. 3150.patch
          75 kB
          Devaraj Das
        2. 3150 test plan.txt
          2 kB
          Amareshwari Sriramadasu
        3. patch-3150.txt
          126 kB
          Amareshwari Sriramadasu
        4. patch-3150.txt
          124 kB
          Amareshwari Sriramadasu
        5. patch-3150.txt
          124 kB
          Amareshwari Sriramadasu
        6. patch-3150.txt
          120 kB
          Amareshwari Sriramadasu
        7. patch-3150.txt
          113 kB
          Amareshwari Sriramadasu
        8. patch-3150.txt
          109 kB
          Amareshwari Sriramadasu
        9. patch-3150.txt
          110 kB
          Amareshwari Sriramadasu
        10. patch-3150.txt
          106 kB
          Amareshwari Sriramadasu
        11. patch-3150.txt
          105 kB
          Amareshwari Sriramadasu
        12. patch-3150.txt
          89 kB
          Amareshwari Sriramadasu
        13. patch-3150.txt
          85 kB
          Amareshwari Sriramadasu
        14. patch-3150.txt
          85 kB
          Amareshwari Sriramadasu
        15. patch-3150.txt
          88 kB
          Amareshwari Sriramadasu
        16. patch-3150.txt
          82 kB
          Amareshwari Sriramadasu
        17. patch-3150.txt
          82 kB
          Amareshwari Sriramadasu
        18. patch-3150.txt
          77 kB
          Amareshwari Sriramadasu
        19. patch-3150.txt
          77 kB
          Amareshwari Sriramadasu
        20. patch-3150.txt
          73 kB
          Amareshwari Sriramadasu
        21. patch-3150.txt
          116 kB
          Amareshwari Sriramadasu
        22. patch-3150.txt
          116 kB
          Amareshwari Sriramadasu
        23. patch-3150.txt
          116 kB
          Amareshwari Sriramadasu
        24. patch-3150.txt
          110 kB
          Amareshwari Sriramadasu
        25. patch-3150.txt
          105 kB
          Amareshwari Sriramadasu

          Issue Links

            Activity

              People

              • Assignee:
                amareshwari Amareshwari Sriramadasu
                Reporter:
                owen.omalley Owen O'Malley
              • Votes:
                0 Vote for this issue
                Watchers:
                11 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: