Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-15083

History Server would OOM due to unlimited TaskUIData in some stages

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.5.2, 1.6.0, 2.0.0
    • 2.0.1, 2.1.0
    • Web UI
    • None

    Description

      History Server will load all tasks in a stage, which would cause memory leak if tasks occupy too many memory.

      In the following example, a single application would consume 1.1G memory of History Sever.

      I think we should limit tasks memory usages by adding spark.ui.retainedTasks.

      Attachments

        1. Screen Shot 2016-05-01 at 3.50.02 PM.png
          252 kB
          Zheng Tan
        2. Screen Shot 2016-05-01 at 3.51.01 PM.png
          790 kB
          Zheng Tan
        3. Screen Shot 2016-05-01 at 3.51.59 PM.png
          449 kB
          Zheng Tan
        4. Screen Shot 2016-05-01 at 3.55.30 PM.png
          366 kB
          Zheng Tan

        Issue Links

          is duplicated by

          Bug - A problem which impairs or prevents the functions of the product. SPARK-16083 spark HistoryServer memory increases until gets killed by OS.

          • Major - Major loss of function.
          • Resolved
          links to

          Web Link [Github] Pull Request #12990 (tankkyo)

          Pull request #14673 [Github] Pull Request #14673 (ajbozarth)

          Pull request #14794 [Github] Pull Request #14794 (ajbozarth)

          Web Link GitHub Pull Request #14794

          Activity

            People

              ajbozarth Alex Bozarth
              zheng.tan Zheng Tan
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: