Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-6763

Shuffle server listen queue is too small

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • None
    • 2.8.0, 2.7.4, 3.0.0-alpha1
    • mrv2
    • None
    • Reviewed

    Description

      ShuffleHandler doesn't specify a listen queue length for the server port, so it ends up getting the default listen queue length of 50. This is too small to handle bursts of shuffle traffic on large clusters. It's also inconsistent with the default Hadoop uses for RPC servers (default=128).

      Attachments

        1. MAPREDUCE-6763.001.patch
          2 kB
          Jason Darrell Lowe

        Issue Links

          is related to

          Sub-task - The sub-task of the issue TEZ-3415 Ability to configure shuffle server listen queue length

          • Major - Major loss of function.
          • Resolved

          Activity

            People

              jlowe Jason Darrell Lowe
              jlowe Jason Darrell Lowe
              Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: