[SPARK-15869] HTTP 500 and NPE on streaming batch details page

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.0.1, 2.1.0
• Component/s: DStreams, Web UI
• Assignee: Shixiong Zhu
• Reporter: Maciej Bryński
• Created: 10/Jun/16 10:16
• Updated: 18/Aug/16 06:07
• Resolved: 01/Aug/16 21:41

Description

When I'm trying to show details of streaming batch I'm getting NPE.

Sample link:
http://127.0.0.1:4040/streaming/batch/?id=1465553700000

Error:

HTTP ERROR 500

Problem accessing /streaming/batch/. Reason:

    Server Error
Caused by:

java.lang.NullPointerException
	at scala.collection.convert.Wrappers$JCollectionWrapper.iterator(Wrappers.scala:59)
	at scala.collection.IterableLike$class.foreach(IterableLike.scala:72)
	at scala.collection.AbstractIterable.foreach(Iterable.scala:54)
	at scala.collection.TraversableLike$class.groupBy(TraversableLike.scala:320)
	at scala.collection.AbstractTraversable.groupBy(Traversable.scala:104)
	at org.apache.spark.streaming.ui.BatchPage.generateJobTable(BatchPage.scala:273)
	at org.apache.spark.streaming.ui.BatchPage.render(BatchPage.scala:358)
	at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:81)
	at org.apache.spark.ui.WebUI$$anonfun$2.apply(WebUI.scala:81)
	at org.apache.spark.ui.JettyUtils$$anon$2.doGet(JettyUtils.scala:83)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:687)
	at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
	at org.spark_project.jetty.servlet.ServletHolder.handle(ServletHolder.java:812)
	at org.spark_project.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:587)
	at org.spark_project.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1127)
	at org.spark_project.jetty.servlet.ServletHandler.doScope(ServletHandler.java:515)
	at org.spark_project.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1061)
	at org.spark_project.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
	at org.spark_project.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:215)
	at org.spark_project.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:97)
	at org.spark_project.jetty.server.Server.handle(Server.java:499)
	at org.spark_project.jetty.server.HttpChannel.handle(HttpChannel.java:311)
	at org.spark_project.jetty.server.HttpConnection.onFillable(HttpConnection.java:257)
	at org.spark_project.jetty.io.AbstractConnection$2.run(AbstractConnection.java:544)
	at org.spark_project.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:635)
	at org.spark_project.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:555)
	at java.lang.Thread.run(Thread.java:745)

Issue Links

• links to: [Github] Pull Request #14443 (zsxwing)