Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-10640

Spark history server fails to parse taskEndReasonFromJson TaskCommitDenied

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 1.4.0, 1.5.0
    • Fix Version/s: 1.5.1, 1.6.0
    • Component/s: Spark Core
    • Labels:
      None

      Description

      I'm seeing an exception from the spark history server trying to read a history file:

      scala.MatchError: TaskCommitDenied (of class java.lang.String)
              at org.apache.spark.util.JsonProtocol$.taskEndReasonFromJson(JsonProtocol.scala:775)
              at org.apache.spark.util.JsonProtocol$.taskEndFromJson(JsonProtocol.scala:531)
              at org.apache.spark.util.JsonProtocol$.sparkEventFromJson(JsonProtocol.scala:488)
              at org.apache.spark.scheduler.ReplayListenerBus.replay(ReplayListenerBus.scala:58)
              at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$replay(FsHistoryProvider.scala:457)
              at org.apache.spark.deploy.history.FsHistoryProvider$$anonfun$10.apply(FsHistoryProvider.scala:292)
              at org.apache.spark.deploy.history.FsHistoryProvider$$anonfun$10.apply(FsHistoryProvider.scala:289)
              at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
              at scala.collection.TraversableLike$$anonfun$flatMap$1.apply(TraversableLike.scala:251)
              at scala.collection.mutable.ResizableArray$class.foreach(ResizableArray.scala:59)
              at scala.collection.mutable.ArrayBuffer.foreach(ArrayBuffer.scala:47)
              at scala.collection.TraversableLike$class.flatMap(TraversableLike.scala:251)
              at scala.collection.AbstractTraversable.flatMap(Traversable.scala:105)
              at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$mergeApplicationListing(FsHistoryProvider.scala:289)
              at org.apache.spark.deploy.history.FsHistoryProvider$$anonfun$checkForLogs$1$$anon$2.run(FsHistoryProvider.scala:210)
              at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
              at java.util.concurrent.FutureTask.run(FutureTask.java:262)
              at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
              at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
              at java.lang.Thread.run(Thread.java:745)
      

        Attachments

          Activity

            People

            • Assignee:
              andrewor14 Andrew Or
              Reporter:
              tgraves Thomas Graves
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: