Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-24677

TaskSetManager not updating successfulTaskDurations for old stage attempts

Log workAgile BoardRank to TopRank to BottomAttach filesAttach ScreenshotVotersStop watchingWatchersCreate sub-taskConvert to sub-taskLinkCloneLabelsUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 2.3.1
    • Fix Version/s: 2.2.3, 2.3.3, 2.4.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      When introducing SPARK-23433 , maybe cause stop sparkcontext.

      ERROR Utils: uncaught error in thread task-scheduler-speculation, stopping SparkContext
      java.util.NoSuchElementException: MedianHeap is empty.
      at org.apache.spark.util.collection.MedianHeap.median(MedianHeap.scala:83)
      at org.apache.spark.scheduler.TaskSetManager.checkSpeculatableTasks(TaskSetManager.scala:968)
      at org.apache.spark.scheduler.Pool$$anonfun$checkSpeculatableTasks$1.apply(Pool.scala:94)
      at org.apache.spark.scheduler.Pool$$anonfun$checkSpeculatableTasks$1.apply(Pool.scala:93)
      at scala.collection.Iterator$class.foreach(Iterator.scala:742)
      at scala.collection.AbstractIterator.foreach(Iterator.scala:1194)
      at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
      at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
      at org.apache.spark.scheduler.Pool.checkSpeculatableTasks(Pool.scala:93)
      at org.apache.spark.scheduler.Pool$$anonfun$checkSpeculatableTasks$1.apply(Pool.scala:94)
      at org.apache.spark.scheduler.Pool$$anonfun$checkSpeculatableTasks$1.apply(Pool.scala:93)
      

        Attachments

          Activity

          $i18n.getText('security.level.explanation', $currentSelection) Viewable by All Users
          Cancel

            People

              Dates

              • Created:
                Updated:
                Resolved:

                Issue deployment