Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6763

Shuffle server listen queue is too small

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      ShuffleHandler doesn't specify a listen queue length for the server port, so it ends up getting the default listen queue length of 50. This is too small to handle bursts of shuffle traffic on large clusters. It's also inconsistent with the default Hadoop uses for RPC servers (default=128).

        Attachments

        1. MAPREDUCE-6763.001.patch
          2 kB
          Jason Darrell Lowe

          Issue Links

            Activity

              People

              • Assignee:
                jlowe Jason Darrell Lowe
                Reporter:
                jlowe Jason Darrell Lowe
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: