XMLWordPrintableJSON

    Details

    • Type: Sub-task
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
    • Target Version/s:
    • Sprint:
      HDDS BadLands

      Description

      Failed in it-hdds-om:

      [INFO] Running org.apache.hadoop.ozone.om.TestKeyPurging
      [INFO] Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 36.933 s - in org.apache.hadoop.ozone.om.TestKeyPurging
      [INFO] Running org.apache.hadoop.ozone.om.TestOzoneManagerRocksDBLogging
      [ERROR] Tests run: 1, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 60.44 s <<< FAILURE! - in org.apache.hadoop.ozone.om.TestOzoneManagerRocksDBLogging
      [ERROR] testOMRocksDBLoggingEnabled(org.apache.hadoop.ozone.om.TestOzoneManagerRocksDBLogging)  Time elapsed: 60.023 s  <<< ERROR!
      java.lang.Exception: test timed out after 60000 milliseconds
      	at java.lang.Object.wait(Native Method)
      	at java.util.concurrent.ForkJoinTask.externalAwaitDone(ForkJoinTask.java:334)
      	at java.util.concurrent.ForkJoinTask.doInvoke(ForkJoinTask.java:405)
      	at java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:734)
      	at java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
      	at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
      	at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
      	at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:485)
      	at java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:650)
      	at org.apache.hadoop.ozone.MiniOzoneClusterImpl.stopDatanodes(MiniOzoneClusterImpl.java:463)
      	at org.apache.hadoop.ozone.MiniOzoneClusterImpl.stop(MiniOzoneClusterImpl.java:404)
      	at org.apache.hadoop.ozone.MiniOzoneClusterImpl.shutdown(MiniOzoneClusterImpl.java:391)
      	at org.apache.hadoop.ozone.om.TestOzoneManagerRocksDBLogging.shutdown(TestOzoneManagerRocksDBLogging.java:63)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      
      [INFO] Results:
      [INFO] 
      [ERROR] Errors: 
      [ERROR]   TestOzoneManagerRocksDBLogging.shutdown:63->Object.wait:-2 ?  test timed out a...
      [INFO] 
      [ERROR] Tests run: 62, Failures: 0, Errors: 1, Skipped: 12
      

      Output is FLOODED with the same message:

      org.apache.hadoop.ozone.om.TestOzoneManagerRocksDBLogging-output.txt
      2020-04-01 20:52:54,072 [Datanode State Machine Thread - 0] ERROR statemachine.DatanodeStateMachine (DatanodeStateMachine.java:start(232)) - Unable to finish the execution.
      java.util.concurrent.RejectedExecutionException: Task java.util.concurrent.ExecutorCompletionService$QueueingFuture@24ae1cc8 rejected from org.apache.hadoop.util.concurrent.HadoopThreadPoolExecutor@24ed6a81[Terminated, pool size = 0, active threads = 0, queued tasks = 0, completed tasks = 35]
      	at java.util.concurrent.ThreadPoolExecutor$AbortPolicy.rejectedExecution(ThreadPoolExecutor.java:2063)
      	at java.util.concurrent.ThreadPoolExecutor.reject(ThreadPoolExecutor.java:830)
      	at java.util.concurrent.ThreadPoolExecutor.execute(ThreadPoolExecutor.java:1379)
      	at java.util.concurrent.ExecutorCompletionService.submit(ExecutorCompletionService.java:181)
      	at org.apache.hadoop.ozone.container.common.states.datanode.RunningDatanodeState.execute(RunningDatanodeState.java:144)
      	at org.apache.hadoop.ozone.container.common.statemachine.StateContext.execute(StateContext.java:419)
      	at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.start(DatanodeStateMachine.java:219)
      	at org.apache.hadoop.ozone.container.common.statemachine.DatanodeStateMachine.lambda$startDaemon$0(DatanodeStateMachine.java:386)
      	at java.lang.Thread.run(Thread.java:748)
      

      Artifact: https://github.com/apache/hadoop-ozone/suites/565485547/artifacts/3737405

      Could be related to HDDS-2866 ?

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              smeng Siyao Meng
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: