Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-9894

CapacitySchedulerPerf test for measuring hundreds of apps in a large number of queues.

    XMLWordPrintableJSON

    Details

      Description

      I have developed a unit test based on the existing TestCapacitySchedulerPerf tests that will measure the performance of a configurable number of apps in a configurable number of queues. It will also test the performance of a cluster that has many queues but only a portion of them are active.

      For example:
      $ mvn test -Dtest=TestCapacitySchedulerPerf#testUserLimitThroughputWithManyQueues \
        -DRunCapacitySchedulerPerfTests=true
        -DNumberOfQueues=100 \
        -DNumberOfApplications=200 \
        -DPercentActiveQueues=100
      
      • Parameters:
        • RunCapacitySchedulerPerfTests=true:
          Needed in order to trigger the test
        • NumberOfQueues
          Configurable number of queues
        • NumberOfApplications
          Total number of apps to run in the whole cluster, distributed evenly across all queues
        • PercentActiveQueues
          Percentage of the queues that contain active applications

        Attachments

        1. YARN-9894.001.patch
          12 kB
          epayne-old
        2. YARN-9894.002.patch
          13 kB
          Eric Payne

          Activity

            People

            • Assignee:
              epayne Eric Payne
              Reporter:
              epayne Eric Payne
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: