Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-16332

the history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Duplicate
    • 2.0.0
    • None
    • Spark Core
    • None
    • this is seen on both x86 (Intel(R) Xeon(R), E5-2699 ) and ppc platform IBM Power8 Habanero (Model: 8348-21C), Red Hat Enterprise Linux Server release 7.2 (Maipo), Spark2.0.0-preview (May-24, 2016)

    • Important

    Description

      the JVM instance of the Spark history server of spark2.0-preview (may-24 build) consumes more than 1000% cpu without the spark standalone cluster (of 6 nodes) being under load.
      When under load (1TB input data for a SQL query scenario), the JVM instance of the Spark history server of spark2.0-preview consumes 2000% cpu (as seen with "top" on linux 3.10)
      Note: can't see a proper Component selection here, surely not a Web GUI issue.

      Attachments

        Issue Links

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            Unassigned Unassigned
            petergangliu Peter Liu
            Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack

                Issue deployment