Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3561 [Umbrella ticket] Performance issues in YARN+MR
  3. MAPREDUCE-3569

TaskAttemptListener holds a global lock for all task-updates

        Details

        • Type: Sub-task Sub-task
        • Status: Closed
        • Priority: Critical Critical
        • Resolution: Fixed
        • Affects Version/s: 0.23.1
        • Fix Version/s: 0.23.1
        • Component/s: mr-am, mrv2, performance
        • Labels:
          None
        • Hadoop Flags:
          Reviewed

          Description

          This got added via MAPREDUCE-3274. We really don't need the lock if we just implement what I mentioned on that ticket here.

          This has performance implications on MR AM with lots of tasks.

          1. Text File
            MAPREDUCE-3569-20111115.1.txt
            12 kB
            Vinod Kumar Vavilapalli
          2. Text File
            MAPREDUCE-3569-20120104.txt
            16 kB
            Vinod Kumar Vavilapalli
          3. Text File
            MR3569.txt
            15 kB
            Siddharth Seth

            Activity

            Ascending order - Click to sort in descending order
            Vinod Kumar Vavilapalli created issue -
            Vinod Kumar Vavilapalli made changes -
            Field Original Value New Value
            Component/s mrv2 [ 12314301 ]
            Component/s performance [ 12316500 ]
            Vinod Kumar Vavilapalli made changes -
            Fix Version/s 0.23.1 [ 12318883 ]
            Affects Version/s 0.23.1 [ 12318883 ]
            Description This got added via MAPREDUCE-3274. We really don't need the lock if we just implement what I mentioned on that ticket [here|https://issues.apache.org/jira/browse/MAPREDUCE-3274?focusedCommentId=13137214&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13137214].

            This has performance implications on MR AM with lots of tasks.
            Component/s mr-am [ 12315342 ]
            Vinod Kumar Vavilapalli made changes -
            Attachment MAPREDUCE-3569-20111115.1.txt [ 12507618 ]
            Mahadev konar made changes -
            Priority Major [ 3 ] Critical [ 2 ]
            Vinod Kumar Vavilapalli made changes -
            Attachment MAPREDUCE-3569-20120104.txt [ 12509458 ]
            Vinod Kumar Vavilapalli made changes -
            Status Open [ 1 ] Patch Available [ 10002 ]
            Siddharth Seth made changes -
            Status Patch Available [ 10002 ] Open [ 1 ]
            Siddharth Seth made changes -
            Attachment MR3569.txt [ 12509501 ]
            Siddharth Seth made changes -
            Status Open [ 1 ] Patch Available [ 10002 ]
            Siddharth Seth made changes -
            Status Patch Available [ 10002 ] Resolved [ 5 ]
            Hadoop Flags Reviewed [ 10343 ]
            Resolution Fixed [ 1 ]
            Arun C Murthy made changes -
            Status Resolved [ 5 ] Closed [ 6 ]

              People

              • Assignee:
                Vinod Kumar Vavilapalli
                Reporter:
                Vinod Kumar Vavilapalli
              • Votes:
                0 Vote for this issue
                Watchers:
                1 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Development