Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.203.0, 1.0.3
    • Fix Version/s: 1.0.4
    • Component/s: performance, tasktracker
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      There is a significant (up to 3x) performance regression in shuffle (vs 0.20.2) in the Hadoop 1.x series. Most noticeable with high-end switches.

        Activity

        Hide
        Matt Foley added a comment -

        Closed upon release of Hadoop-1.0.4.

        Show
        Matt Foley added a comment - Closed upon release of Hadoop-1.0.4.
        Hide
        Matt Foley added a comment -

        Committed to branch-1.1 and branch-1.0. Thanks, Luke!

        Show
        Matt Foley added a comment - Committed to branch-1.1 and branch-1.0. Thanks, Luke!
        Hide
        Luke Lu added a comment -

        @Suresh, 0.20.2 use jetty 6.1.14, which doesn't have this particular issue. This is a workaround specifically for jetty 6.1.26 and harmless for other versions.

        Show
        Luke Lu added a comment - @Suresh, 0.20.2 use jetty 6.1.14, which doesn't have this particular issue. This is a workaround specifically for jetty 6.1.26 and harmless for other versions.
        Hide
        Suresh Srinivas added a comment -

        Nicholas, can you please verify if this is an issue for WebHDFS (both on 1.0 and trunk?)

        Show
        Suresh Srinivas added a comment - Nicholas, can you please verify if this is an issue for WebHDFS (both on 1.0 and trunk?)
        Hide
        Suresh Srinivas added a comment -

        I committed the patch to branch-1 and for 1.1.0.

        Thank you Luke for the patch. Thank you John Poelman and Shreyas Subramanya for noticing the issue and verifying the fix.

        Show
        Suresh Srinivas added a comment - I committed the patch to branch-1 and for 1.1.0. Thank you Luke for the patch. Thank you John Poelman and Shreyas Subramanya for noticing the issue and verifying the fix.
        Hide
        Suresh Srinivas added a comment -

        Given the code is the same, this may not be regression compared to 0.20.2. However, this is straightforward change, and given the link you posted, I am +1 on committing this.

        Show
        Suresh Srinivas added a comment - Given the code is the same, this may not be regression compared to 0.20.2. However, this is straightforward change, and given the link you posted, I am +1 on committing this.
        Hide
        Luke Lu added a comment -

        The crux of the problem is jetty 6.1.26's internal buffer management with transfer-encoding = chunked. The issue was also independently discovered and analyzed elsewhere: http://goo.gl/Wln7Z

        Show
        Luke Lu added a comment - The crux of the problem is jetty 6.1.26's internal buffer management with transfer-encoding = chunked. The issue was also independently discovered and analyzed elsewhere: http://goo.gl/Wln7Z
        Hide
        Luke Lu added a comment -

        Thanks to John Poelman and Shreyas Subramanya of IBM BigInsights performance QA for noticing the issue and verifying my fix.

        Show
        Luke Lu added a comment - Thanks to John Poelman and Shreyas Subramanya of IBM BigInsights performance QA for noticing the issue and verifying my fix.

          People

          • Assignee:
            Luke Lu
            Reporter:
            Luke Lu
          • Votes:
            0 Vote for this issue
            Watchers:
            14 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development