Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-2964

Fix @Ignore-d integration tests

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • None
    • None
    • test

    Description

      We marked all the intermittent unit tests with @Ignore to get reliable feedback from CI builds.

      Before HDDS-2833 we had 21 @Ignore annotations, HDDS-2833 introduced 34 new one.

      We need to review all of these tests and either fix, or delete or convert them to real unit tests.

      The current list of ignore tests:

      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestContainerPlacement.java:  @Ignore
      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java:  @Ignore("Tracked by HDDS-2508.")
      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java:  @Ignore
      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/container/TestContainerStateManagerIntegration.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/container/TestContainerStateManagerIntegration.java:  @Ignore("TODO:HDDS-1159")
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/pipeline/TestNodeFailure.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/pipeline/TestNodeFailure.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineCreateAndDestroy.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/safemode/TestSCMSafeModeWithPipelineRules.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/Test2WayCommitInRatis.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestBlockOutputStreamWithFailures.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestCloseContainerHandlingByClient.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestCloseContainerHandlingByClient.java:  @Ignore // test needs to be fixed after close container is handled for
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestCommitWatcher.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestContainerReplicationEndToEnd.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestContainerStateMachineFailures.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestContainerStateMachine.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestDeleteWithSlowFollower.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestFailureHandlingByClient.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestMultiBlockWritesWithDnFailures.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneAtRestEncryption.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneClientRetriesOnException.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientAbstract.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientAbstract.java:  @Ignore("Debug Jenkins Timeout")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientForAclAuditLog.java:@Ignore("Fix this after adding audit support for HA Acl code. This will be " +
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientWithRatis.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestSecureOzoneRpcClient.java:  @Ignore("Needs to be moved out of this class as  client setup is static")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestBlockDeletion.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestCloseContainerByPipeline.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/common/transport/server/ratis/TestCSMMetrics.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainer.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java:@Ignore("Disabling Ratis tests for pipeline work.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java:@Ignore("TODO:HDDS-1157")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestRatisManager.java:@Ignore("Disabling Ratis tests for pipeline work.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/server/TestContainerServer.java:@Ignore("Takes too long to run this test. Ignoring for time being.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/TestContainerReplication.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/freon/TestFreonWithDatanodeFastRestart.java:  @Ignore("TODO:HDDS-1160")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/freon/TestFreonWithDatanodeRestart.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/freon/TestFreonWithPipelineDestroy.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOmMetrics.java:  @Ignore("Test failing because of table cache. Revisit later.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOMRatisSnapshots.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOmSQLCli.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOzoneManagerHA.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOzoneManagerRestart.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestScmSafeMode.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/node/TestQueryNode.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/pipeline/TestSCMPipelineMetrics.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/TestGetCommittedBlockLengthAndPutKey.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/TestSCMContainerPlacementPolicyMetrics.java:@Ignore("HDDS-2961")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/TestMiniOzoneCluster.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/TestStorageContainerManager.java:@Ignore
      hadoop-ozone/ozone-manager org/apache/hadoop/ozone/om/ratis/TestOzoneManagerDoubleBufferWithOMResponse.java:@Ignore("HDDS-2648")
      hadoop-ozone/ozone-manager org/apache/hadoop/ozone/om/TestBucketManagerImpl.java:@Ignore("Bucket Manager does not use cache, Disable it for now.") 

       

      Attachments

        Issue Links

          1.
          Fix TestNodeFailure.java Sub-task Open Mukul Kumar Singh  
          2.
          Fix TestCommitWatcher.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m
          3.
          Fix TestBlockOutputStreamWithFailures.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          4.
          Fix TestBlockOutputStreamWithFailures.java Sub-task Resolved Shashikant Banerjee  
          5.
          Fix Test2WayCommitInRatis.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 10m
          6.
          Fix TestMultiBlockWritesWithDnFailures.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          7.
          Fix TestGetCommittedBlockLengthAndPutKey.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          8.
          Fix TestContainerStateMachineFailures.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 50m
          9.
          Fix TestContainerStateMachine.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          10.
          Fix TestOzoneClientRetriesOnException.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          11.
          Remove TestContainerPlacement add comprehensive unit level tests to container placement policies Sub-task Open István Fajth  
          12.
          Fix TestFailureHandlingByClient.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          13.
          Fix TestDeleteWithSlowFollower.java Sub-task Resolved Shashikant Banerjee

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 40m
          14.
          Fix TestOzoneRpcClientAbstract.testPutKeyRatisThreeNodesParallel Sub-task Open Marton Elek  
          15.
          Fix TestOzoneRpcClientAbstract.testListVolume Sub-task Resolved Lokesh Jain  
          16.
          TestRatisPipelineProvider failure Sub-task Resolved Bharat Viswanadham  
          17.
          TestOzoneManagerDoubleBufferWithOMResponse Sub-task Resolved Bharat Viswanadham

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          18.
          Fix TestDeadNodeHandler. Sub-task Resolved Aravindan Vijayan

          100%

          Original Estimate - Not Specified Original Estimate - Not Specified
          Time Spent - 20m
          19.
          Fix TestFreonWithPipelineDestroy.java Sub-task Open Unassigned  
          20.
          Fix TestOzoneManagerRestart.java Sub-task Open Peter Orova  
          21.
          Fix TestContainerReplicationEndToEnd.java Sub-task Resolved Peter Orova  
          22.
          Fix TestScmSafeMode Sub-task Open Unassigned  
          23.
          Fix TestOzoneRpcClientWithRatis.java Sub-task Resolved Shashikant Banerjee  
          24.
          Fix TestCSMMetrics.java Sub-task Resolved Shashikant Banerjee  
          25.
          Fix TestReadRetries Sub-task Resolved Shashikant Banerjee  
          26.
          Fix TestOzoneManagerHA#testIncrementalWaitTimeWithSameNodeFailover Sub-task Open Unassigned  
          27.
          Fix TestOzoneManagerHA#testMultipartUploadWithOneOmNodeDown Sub-task Open Unassigned  
          28.
          Fix TestBlockOutputStreamWithFailures#test2DatanodesFailure Sub-task Resolved Shashikant Banerjee  
          29.
          Fix TestOzoneRpcClientAbstract#testDeletedKeyForGDPR Sub-task Resolved Lokesh Jain  

          Activity

            People

              Unassigned Unassigned
              elek Marton Elek
              Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

                Created:
                Updated:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 5h
                  5h