Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.21.0
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Adds Reduce Attempt ID to ClientTrace log messages, and adds Reduce Attempt ID to HTTP query string sent to mapOutputServlet. Extracts partition number from attempt ID.

      Description

      Current clienttrace messages from shuffles note only the destination map ID but not the source reduce ID. Having both source and destination ID of each shuffle enables full tracing of execution.

      1. MAPREDUCE-479-4.patch
        3 kB
        Jiaqi Tan
      2. MAPREDUCE-479-3.patch
        3 kB
        Jiaqi Tan
      3. MAPREDUCE-479-2.patch
        3 kB
        Jiaqi Tan
      4. MAPREDUCE-479-1.patch
        7 kB
        Jiaqi Tan
      5. MAPREDUCE-479.patch
        2 kB
        Jiaqi Tan
      6. HADOOP-6013.patch
        1 kB
        Jiaqi Tan

        Issue Links

          Activity

          Tom White made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags [Incompatible change, Reviewed]
          Resolution Fixed [ 1 ]
          Jiaqi Tan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Adds Reduce Attempt ID to ClientTrace log messages, and adds Reduce Attempt ID to HTTP query string sent to mapOutputServlet. Adds Reduce Attempt ID to ClientTrace log messages, and adds Reduce Attempt ID to HTTP query string sent to mapOutputServlet. Extracts partition number from attempt ID.
          Jiaqi Tan made changes -
          Attachment MAPREDUCE-479-4.patch [ 12415747 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jiaqi Tan made changes -
          Attachment MAPREDUCE-479-3.patch [ 12413440 ]
          Jiaqi Tan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jiaqi Tan made changes -
          Attachment MAPREDUCE-479-2.patch [ 12413324 ]
          Jiaqi Tan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jiaqi Tan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Adds Reduce ID to ClientTrace log messages. Explicitly uses new mapreduce.JobID for compatibility with updated TaskID constructor. Adds Reduce Attempt ID to ClientTrace log messages, and adds Reduce Attempt ID to HTTP query string sent to mapOutputServlet.
          Jiaqi Tan made changes -
          Attachment MAPREDUCE-479-1.patch [ 12413322 ]
          Jiaqi Tan made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jiaqi Tan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Release Note Adds Reduce ID to ClientTrace log messages. Explicitly uses new mapreduce.JobID for compatibility with updated TaskID constructor.
          Affects Version/s 0.21.0 [ 12314045 ]
          Fix Version/s 0.21.0 [ 12314045 ]
          Jiaqi Tan made changes -
          Attachment MAPREDUCE-479.patch [ 12413251 ]
          Chris Douglas made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Assignee Jiaqi Tan [ tanjiaqi ]
          Owen O'Malley made changes -
          Project Hadoop Common [ 12310240 ] Hadoop Map/Reduce [ 12310941 ]
          Key HADOOP-6013 MAPREDUCE-479
          Affects Version/s 0.20.0 [ 12313438 ]
          Component/s mapred [ 12310690 ]
          Fix Version/s 0.21.0 [ 12313563 ]
          Jiaqi Tan made changes -
          Link This issue relates to HADOOP-5876 [ HADOOP-5876 ]
          Jiaqi Tan made changes -
          Priority Major [ 3 ] Minor [ 4 ]
          Jiaqi Tan made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jiaqi Tan made changes -
          Field Original Value New Value
          Attachment HADOOP-6013.patch [ 12410427 ]
          Jiaqi Tan created issue -

            People

            • Assignee:
              Jiaqi Tan
              Reporter:
              Jiaqi Tan
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development