Uploaded image for project: 'Hadoop YARN'
  1. Hadoop YARN
  2. YARN-2014

Performance: AM scaleability is 10% slower in 2.4 compared to 0.23.9

Add voteVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 2.4.0
    • None
    • None
    • None

    Description

      Performance comparison benchmarks from 2.x against 0.23 shows AM scalability benchmark's runtime is approximately 10% slower in 2.4.0. The trend is consistent across later releases in both lines, latest release numbers are:

      2.4.0.0 runtime 255.6 seconds (avg 5 passes)
      0.23.9.12 runtime 230.4 seconds (avg 5 passes)
      Diff: -9.9%

      AM Scalability test is essentially a sleep job that measures time to launch and complete a large number of mappers.

      The diff is consistent and has been reproduced in both a larger (350 node, 100,000 mappers) perf environment, as well as a small (10 node, 2,900 mappers) demo cluster.

      Attachments

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            jlowe Jason Darrell Lowe
            patwhitey2007 patrick white

            Dates

              Created:
              Updated:

              Slack

                Issue deployment