Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-4072

Fix CreateNote() function throw Note existed IOException caused travis to fail

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None

      Description

      When using trivis-ci, The exception will be triggered randomly.

      01:52:28,183  INFO org.apache.zeppelin.storage.LocalConfigStorage:74 - Save notebook authorization to file: /home/travis/build/liuxunorg/zeppelin/zeppelin-server/../conf_RecoveryTest/notebook-authorization.json
      01:52:28,192  INFO org.apache.zeppelin.notebook.repo.VFSNotebookRepo:140 - Saving note 2E5FGMPF9 to note1_2E5FGMPF9.zpln
      01:52:28,481  INFO org.apache.zeppelin.notebook.repo.VFSNotebookRepo:140 - Saving note 2E5FGMPF9 to note1_2E5FGMPF9.zpln
      java.io.IOException: Note '/note1' existed
      	at org.apache.zeppelin.notebook.NoteManager.addOrUpdateNoteNode(NoteManager.java:117)
      	at org.apache.zeppelin.notebook.NoteManager.addNote(NoteManager.java:171)
      	at org.apache.zeppelin.notebook.Notebook.createNote(Notebook.java:186)
      	at org.apache.zeppelin.notebook.Notebook.createNote(Notebook.java:167)
      	at org.apache.zeppelin.socket.NotebookServerTest.testGetParagraphList(NotebookServerTest.java:644)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50)
      	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47)
      	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78)
      	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57)
      	at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290)
      	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71)
      	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288)
      	at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58)
      	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268)
      	at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26)
      	at org.junit.internal.runners.statements.RunAfters.evaluate(RunAfters.java:27)
      	at org.junit.runners.ParentRunner.run(ParentRunner.java:363)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
      	at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
      	at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
      	at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
      	at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
      01:52:28,550  INFO org.apache.zeppelin.socket.NotebookServer:150 - NotebookServer instantiated: org.apache.zeppelin.socket.NotebookServer@622fdb81
      01:52:28,551  INFO org.apache.zeppelin.socket.NotebookServer:161 - Injected NotebookProvider
      01:52:28,552  INFO org.apache.zeppelin.socket.NotebookServer:168 - Injected NotebookServiceProvider
      

      I think it may be caused by several situations.
      There are multiple test units in a test class.
      Each unit creates a note with the same name and then deletes it.
      If the test unit performs an exception and does not delete the note normally, it will cause other test units to create a note exception with the same name.

      1) My simple test to create a note and then delete, are normal,
      In ZeppelinSparkClusterTest24, problems may be caused by the operation of spark.

      ============================================================

      Cause Analysis:
      1. Recently, Notebook added a new one in createNote() to determine if the note name already exists. If it exists, an exception is thrown.
      2. In the test case, the code
      TestUtils.getInstance(Notebook.class) creates a global single instance object.
      First executed:
      TestUtils.getInstance(Notebook.class).createNote("note1", anonymous);
      After executing some test code, eg: spark, python, etc
      TestUtils.getInstance(Notebook.class).removeNote(note.getId(), anonymous);
      Delete this note.
      If the spark, python exception is thrown before removeNote, then the note still exists. Causes other test cases to be abnormal.

      So in the test code, make sure the note is removed by try(){}finally.

        Attachments

        1. WechatIMG2.png
          147 kB
          Xun Liu
        2. WechatIMG1.png
          311 kB
          Xun Liu

          Issue Links

            Activity

              People

              • Assignee:
                liuxun323 Xun Liu
                Reporter:
                liuxun323 Xun Liu
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 40m
                  40m