Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-4703

"Expected BEGIN_OBJECT but was STRING at line 1 column xx" pop up from time to time

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: Blocker
    • Resolution: Unresolved
    • Affects Version/s: 0.9.0
    • Fix Version/s: None
    • Component/s: zeppelin-server
    • Labels:
      None
    • Environment:

      zeppelin version: 8988723

      k8s mode

      spark 2.4.6-SNAPSHOT

      Scala: 2.11

       

      Description

      "Expected BEGIN_OBJECT but was STRING at line 1 column xx" pop up from time to time and cannot use zeppelin at all. The xx number means it can be any number

       

      logs:
      ERROR [2020-03-30 03:09:54,147] ({qtp1273765644-16} NotebookServer.java[onMessage]:459) - Can't handle message: 7e0c3d2Pmin"]"principal":"admin","ticket":"34130484-5e68-48f1-8e10-17e0c3d2397d","roles":"[\"admin\"]"}
      com.google.gson.JsonSyntaxException: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 13
      at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:176)
      at com.google.gson.Gson.fromJson(Gson.java:791)
      at com.google.gson.Gson.fromJson(Gson.java:757)
      at com.google.gson.Gson.fromJson(Gson.java:706)
      at com.google.gson.Gson.fromJson(Gson.java:678)
      at org.apache.zeppelin.socket.NotebookServer.deserializeMessage(NotebookServer.java:477)
      at org.apache.zeppelin.socket.NotebookServer.onMessage(NotebookServer.java:250)
      at org.apache.zeppelin.socket.NotebookSocket.onWebSocketText(NotebookSocket.java:58)
      at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextMessage(JettyListenerEventDriver.java:189)
      at org.eclipse.jetty.websocket.common.message.SimpleTextMessage.messageComplete(SimpleTextMessage.java:69)
      at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.appendMessage(AbstractEventDriver.java:65)
      at org.eclipse.jetty.websocket.common.events.JettyListenerEventDriver.onTextFrame(JettyListenerEventDriver.java:158)
      at org.eclipse.jetty.websocket.common.events.AbstractEventDriver.incomingFrame(AbstractEventDriver.java:150)
      at org.eclipse.jetty.websocket.common.WebSocketSession.incomingFrame(WebSocketSession.java:319)
      at org.eclipse.jetty.websocket.common.extensions.AbstractExtension.nextIncomingFrame(AbstractExtension.java:147)
      at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.nextIncomingFrame(PerMessageDeflateExtension.java:112)
      at org.eclipse.jetty.websocket.common.extensions.compress.CompressExtension.forwardIncoming(CompressExtension.java:142)
      at org.eclipse.jetty.websocket.common.extensions.compress.PerMessageDeflateExtension.incomingFrame(PerMessageDeflateExtension.java:92)
      at org.eclipse.jetty.websocket.common.extensions.ExtensionStack.incomingFrame(ExtensionStack.java:201)
      at org.eclipse.jetty.websocket.common.Parser.notifyFrame(Parser.java:226)
      at org.eclipse.jetty.websocket.common.Parser.parse(Parser.java:255)
      at
      org.eclipse.jetty.websocket.common.io
      .AbstractWebSocketConnection.readParse(AbstractWebSocketConnection.java:581)
      at
      org.eclipse.jetty.websocket.common.io
      .AbstractWebSocketConnection.onFillable(AbstractWebSocketConnection.java:434)
      at
      org.eclipse.jetty.io
      .AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:305)
      at
      org.eclipse.jetty.io
      .FillInterest.fillable(FillInterest.java:103)
      at
      org.eclipse.jetty.io
      .ChannelEndPoint$2.run(ChannelEndPoint.java:117)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.runTask(EatWhatYouKill.java:333)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:310)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.tryProduce(EatWhatYouKill.java:168)
      at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:126)
      at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:366)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:698)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$Runner.run(QueuedThreadPool.java:804)
      at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.IllegalStateException: Expected BEGIN_OBJECT but was STRING at line 1 column 13
      at com.google.gson.stream.JsonReader.expect(JsonReader.java:339)
      at com.google.gson.stream.JsonReader.beginObject(JsonReader.java:322)
      at com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:165)
      ... 33 more
       

      some observations: 

      1. we suspect it may be caused by imcompatible between old notebook and new zeppelin version so we deleted all old notebooks, and create a new notebook, but the issue still be there.
      2. looks like when we delete file notebook-authorization.json, the pop up message disapeared, but i'm not sure about this.

       

      If this is a known issue and need long time to fix, could you let me know a good version to use. Now it's a blocker issue for us because we cannot use zeppelin in k8s mode.

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              mashuai_191 Shuai Ma
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated: