Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1830

Fix race condition in HdfsServiceTracker

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.10.0
    • Fix Version/s: 0.11.0, 0.12.0
    • Component/s: TajoMaster, Unit Test
    • Labels:
      None

      Description

      If HdfsServiceTracker reads active-master file before the file not closed, active-master file is empty.

      {format}
      testAutoFailOver(org.apache.tajo.ha.TestHAServiceHDFSImpl) Time elapsed: 4.668 sec <<< ERROR!
      org.apache.tajo.exception.TajoRuntimeException: org.apache.tajo.client.v2.exception.ClientConnectionException: java.io.EOFException
      at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:141)
      at org.apache.tajo.client.SessionConnection.<init>(SessionConnection.java:113)
      at org.apache.tajo.client.TajoClientImpl.<init>(TajoClientImpl.java:62)
      at org.apache.tajo.client.TajoClientImpl.<init>(TajoClientImpl.java:86)
      at org.apache.tajo.client.TajoClientImpl.<init>(TajoClientImpl.java:82)
      at org.apache.tajo.ha.TestHAServiceHDFSImpl.verifyDataBaseAndTable(TestHAServiceHDFSImpl.java:152)
      at org.apache.tajo.ha.TestHAServiceHDFSImpl.testAutoFailOver(TestHAServiceHDFSImpl.java:82)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      at java.lang.reflect.Method.invoke(Method.java:606)
      at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:47)
      at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
      at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:44)
      at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
      at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:271)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:70)
      at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:50)
      at org.junit.runners.ParentRunner$3.run(ParentRunner.java:238)
      at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:63)
      at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:236)
      at org.junit.runners.ParentRunner.access$000(ParentRunner.java:53)
      at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:229)
      at org.junit.runners.ParentRunner.run(ParentRunner.java:309)
      at org.apache.maven.surefire.junit4.JUnit4Provider.execute(JUnit4Provider.java:264)
      at org.apache.maven.surefire.junit4.JUnit4Provider.executeTestSet(JUnit4Provider.java:153)
      at org.apache.maven.surefire.junit4.JUnit4Provider.invoke(JUnit4Provider.java:124)
      at org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:200)
      at org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:153)
      at org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)
      Caused by: org.apache.tajo.client.v2.exception.ClientConnectionException: java.io.EOFException
      ... 30 more
      Caused by: org.apache.tajo.service.ServiceTrackerException: java.io.EOFException
      at org.apache.tajo.ha.HdfsServiceTracker.getAddressElements(HdfsServiceTracker.java:508)
      at org.apache.tajo.ha.HdfsServiceTracker.getClientServiceAddress(HdfsServiceTracker.java:409)
      at org.apache.tajo.client.SessionConnection.getTajoMasterAddr(SessionConnection.java:361)
      at org.apache.tajo.client.SessionConnection.getTajoMasterConnection(SessionConnection.java:130)
      ... 29 more
      Caused by: java.io.EOFException
      at java.io.DataInputStream.readUnsignedShort(DataInputStream.java:340)
      at java.io.DataInputStream.readUTF(DataInputStream.java:589)
      at java.io.DataInputStream.readUTF(DataInputStream.java:564)
      at org.apache.tajo.ha.HdfsServiceTracker.getAddressElements(HdfsServiceTracker.java:498)
      ... 32 more
      at org.apache.tajo.ha.HdfsServiceTracker.createMasterFile(HdfsServiceTracker.java:249)
      at org.apache.tajo.ha.HdfsServiceTracker.register(HdfsServiceTracker.java:155)
      at org.apache.tajo.ha.HdfsServiceTracker$PingChecker.run(HdfsServiceTracker.java:374)
      at java.lang.Thread.run(Thread.java:745){format}

        Attachments

          Activity

            People

            • Assignee:
              jhkim Jinho Kim
              Reporter:
              jhkim Jinho Kim
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: