Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6763

Shuffle server listen queue is too small

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.4, 3.0.0-alpha1
    • Component/s: mrv2
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      ShuffleHandler doesn't specify a listen queue length for the server port, so it ends up getting the default listen queue length of 50. This is too small to handle bursts of shuffle traffic on large clusters. It's also inconsistent with the default Hadoop uses for RPC servers (default=128).

        Attachments

        1. MAPREDUCE-6763.001.patch
          2 kB
          Jason Lowe

          Issue Links

            Activity

              People

              • Assignee:
                jlowe Jason Lowe
                Reporter:
                jlowe Jason Lowe
              • Votes:
                0 Vote for this issue
                Watchers:
                9 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: