Uploaded image for project: 'Bigtop'
  1. Bigtop
  2. BIGTOP-1403

Add YARN-2410 to Longevity tests.

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: backlog
    • Fix Version/s: None
    • Component/s: tests
    • Labels:
      None

      Description

      YARN-2410 highlights an interesting longevity bug, wherein it appears if there are large # of reducers ( > 6K )+ 40 map outputs per node, reducers all asking for mapper node outputs at a similar time can lead to scheduler crashing.

      Would be great to see a workload that simulates this in the longevity tests.

        Attachments

          Issue Links

          incorporates

          Bug - A problem which impairs or prevents the functions of the product. MAPREDUCE-6474 ShuffleHandler can possibly exhaust nodemanager file descriptors

          • Major - Major loss of function.
          • Closed

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                jayunit100 jay vyas
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated: