Uploaded image for project: 'Spark'
  1. Spark
  2. SPARK-23139

Read eventLog file with mixed encodings

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Incomplete
    • 2.2.0
    • None
    • Spark Core

    Description

      EventLog may contain mixed encodings such as custom exception message, but caused to replay failure.

      java.nio.charset.MalformedInputException: Input length = 1

      at java.nio.charset.CoderResult.throwException(CoderResult.java:281)
      at sun.nio.cs.StreamDecoder.implRead(StreamDecoder.java:339)
      at sun.nio.cs.StreamDecoder.read(StreamDecoder.java:178)
      at java.io.InputStreamReader.read(InputStreamReader.java:184)
      at java.io.BufferedReader.fill(BufferedReader.java:161)
      at java.io.BufferedReader.readLine(BufferedReader.java:324)
      at java.io.BufferedReader.readLine(BufferedReader.java:389)
      at scala.io.BufferedSource$BufferedLineIterator.hasNext(BufferedSource.scala:72)
      at scala.collection.Iterator$$anon$21.hasNext(Iterator.scala:836)
      at scala.collection.Iterator$$anon$13.hasNext(Iterator.scala:461)
      at org.apache.spark.scheduler.ReplayListenerBus.replay(ReplayListenerBus.scala:78)
      at org.apache.spark.scheduler.ReplayListenerBus.replay(ReplayListenerBus.scala:58)
      at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$replay(FsHistoryProvider.scala:694)
      at org.apache.spark.deploy.history.FsHistoryProvider.org$apache$spark$deploy$history$FsHistoryProvider$$mergeApplicationListing(FsHistoryProvider.scala:507)
      at org.apache.spark.deploy.history.FsHistoryProvider$$anonfun$checkForLogs$4$$anon$4.run(FsHistoryProvider.scala:399)
      at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      at java.lang.Thread.run(Thread.java:745)

      Attachments

        Activity

          People

            Unassigned Unassigned
            Deng FEI DENG FEI
            Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: