[LIVY-619] Getting java.io.IOException: Stream closed very regularly in livy logs

• Type: Bug
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 0.6.0
• Component/s: Server
• Assignee: Unassigned
• Reporter: Parveen
• Created: 01/Aug/19 16:29
• Updated: 01/Aug/19 20:56

Description

We are using livy submitting spark applications through Livys rest interface. Getting this exception quite frequently :

Exception in thread "Thread-110709" java.io.IOException: Stream closed
at java.io.BufferedInputStream.getBufIfOpen(BufferedInputStream.java:170)
at java.io.BufferedInputStream.read1(BufferedInputStream.java:283)
at java.io.BufferedInputStream.read(BufferedInputStream.java:345)
at sun.nio.cs.StreamDecoder.readBytes(StreamDecoder.java:284)
at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:326)
at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
at java.io.InputStreamReader.read(InputStreamReader.java:184)
at java.io.BufferedReader.fill(BufferedReader.java:161)
at java.io.BufferedReader.readLine(BufferedReader.java:324)
at java.io.BufferedReader.readLine(BufferedReader.java:389)
at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:67)
at scala.collection.Iterator$class.foreach(Iterator.scala:727)
at scala.collection.AbstractIterator.foreach(Iterator.scala:1157)
at org.apache.livy.utils.LineBufferedStream$$anon$1.run(LineBufferedStream.scala:40)

 

 

Stack is almost same in almost all of the circumstances. 

 

Have gone through the code and it seems this is harmless as this code only prints logs after getting it from yarn but we are getting it with a very high frequency.