Uploaded image for project: 'Apache Ozone'
  1. Apache Ozone
  2. HDDS-2964

Fix @Ignore-d integration tests

Attach filesAttach ScreenshotVotersWatch issueWatchersCreate sub-taskLinkCloneUpdate Comment AuthorReplace String in CommentUpdate Comment VisibilityDelete Comments
    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Done
    • None
    • 1.4.0
    • test

    Description

      We marked all the intermittent unit tests with @Ignore to get reliable feedback from CI builds.

      Before HDDS-2833 we had 21 @Ignore annotations, HDDS-2833 introduced 34 new one.

      We need to review all of these tests and either fix, or delete or convert them to real unit tests.

      The current list of ignore tests:

      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestContainerPlacement.java:  @Ignore
      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestDeadNodeHandler.java:  @Ignore("Tracked by HDDS-2508.")
      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java:  @Ignore
      hadoop-hdds/server-scm org/apache/hadoop/hdds/scm/node/TestSCMNodeManager.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/container/TestContainerStateManagerIntegration.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/container/TestContainerStateManagerIntegration.java:  @Ignore("TODO:HDDS-1159")
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/pipeline/TestNodeFailure.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/pipeline/TestNodeFailure.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/pipeline/TestRatisPipelineCreateAndDestroy.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/hdds/scm/safemode/TestSCMSafeModeWithPipelineRules.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/Test2WayCommitInRatis.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestBlockOutputStreamWithFailures.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestCloseContainerHandlingByClient.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestCloseContainerHandlingByClient.java:  @Ignore // test needs to be fixed after close container is handled for
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestCommitWatcher.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestContainerReplicationEndToEnd.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestContainerStateMachineFailures.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestContainerStateMachine.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestDeleteWithSlowFollower.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestFailureHandlingByClient.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestMultiBlockWritesWithDnFailures.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneAtRestEncryption.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneClientRetriesOnException.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientAbstract.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientAbstract.java:  @Ignore("Debug Jenkins Timeout")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientForAclAuditLog.java:@Ignore("Fix this after adding audit support for HA Acl code. This will be " +
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestOzoneRpcClientWithRatis.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestSecureOzoneRpcClient.java:  @Ignore("Needs to be moved out of this class as  client setup is static")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/client/rpc/TestWatchForCommit.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestBlockDeletion.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/common/statemachine/commandhandler/TestCloseContainerByPipeline.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/common/transport/server/ratis/TestCSMMetrics.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainer.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerRatis.java:@Ignore("Disabling Ratis tests for pipeline work.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestOzoneContainerWithTLS.java:@Ignore("TODO:HDDS-1157")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/ozoneimpl/TestRatisManager.java:@Ignore("Disabling Ratis tests for pipeline work.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/server/TestContainerServer.java:@Ignore("Takes too long to run this test. Ignoring for time being.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/server/TestSecureContainerServer.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/container/TestContainerReplication.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/freon/TestFreonWithDatanodeFastRestart.java:  @Ignore("TODO:HDDS-1160")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/freon/TestFreonWithDatanodeRestart.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/freon/TestFreonWithPipelineDestroy.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOmMetrics.java:  @Ignore("Test failing because of table cache. Revisit later.")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOMRatisSnapshots.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOmSQLCli.java:  @Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOzoneManagerHA.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestOzoneManagerRestart.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/om/TestScmSafeMode.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/node/TestQueryNode.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/pipeline/TestSCMPipelineMetrics.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/TestGetCommittedBlockLengthAndPutKey.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/scm/TestSCMContainerPlacementPolicyMetrics.java:@Ignore("HDDS-2961")
      hadoop-ozone/integration-test org/apache/hadoop/ozone/TestMiniOzoneCluster.java:@Ignore
      hadoop-ozone/integration-test org/apache/hadoop/ozone/TestStorageContainerManager.java:@Ignore
      hadoop-ozone/ozone-manager org/apache/hadoop/ozone/om/ratis/TestOzoneManagerDoubleBufferWithOMResponse.java:@Ignore("HDDS-2648")
      hadoop-ozone/ozone-manager org/apache/hadoop/ozone/om/TestBucketManagerImpl.java:@Ignore("Bucket Manager does not use cache, Disable it for now.") 

       

      Attachments

        Issue Links

        1.
        Fix TestNodeFailure.java Sub-task Resolved Mukul Kumar Singh   Actions
        2.
        Fix TestCommitWatcher.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        Actions
        3.
        Fix TestBlockOutputStreamWithFailures.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        4.
        Fix TestBlockOutputStreamWithFailures.java Sub-task Resolved Shashikant Banerjee   Actions
        5.
        Fix Test2WayCommitInRatis.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 10m
        Actions
        6.
        Fix TestMultiBlockWritesWithDnFailures.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        7.
        Fix TestGetCommittedBlockLengthAndPutKey.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        8.
        Fix TestContainerStateMachineFailures.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 50m
        Actions
        9.
        Fix TestContainerStateMachine.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        10.
        Fix TestOzoneClientRetriesOnException.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        11.
        Fix TestFailureHandlingByClient.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        12.
        Fix TestDeleteWithSlowFollower.java Sub-task Resolved Shashikant Banerjee

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 40m
        Actions
        13.
        Fix TestOzoneRpcClientAbstract.testPutKeyRatisThreeNodesParallel Sub-task Resolved Marton Elek   Actions
        14.
        Fix TestOzoneRpcClientAbstract.testListVolume Sub-task Resolved Lokesh Jain   Actions
        15.
        TestRatisPipelineProvider failure Sub-task Resolved Bharat Viswanadham   Actions
        16.
        TestOzoneManagerDoubleBufferWithOMResponse Sub-task Resolved Bharat Viswanadham

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        17.
        Fix TestDeadNodeHandler. Sub-task Resolved Aravindan Vijayan

        100%

        Original Estimate - Not Specified Original Estimate - Not Specified
        Time Spent - 20m
        Actions
        18.
        Fix TestFreonWithPipelineDestroy.java Sub-task Resolved Unassigned   Actions
        19.
        Fix TestOzoneManagerRestart.java Sub-task Resolved Peter Orova   Actions
        20.
        Fix TestContainerReplicationEndToEnd.java Sub-task Resolved Peter Orova   Actions
        21.
        Fix TestOzoneRpcClientWithRatis.java Sub-task Resolved Shashikant Banerjee   Actions
        22.
        Fix TestCSMMetrics.java Sub-task Resolved Shashikant Banerjee   Actions
        23.
        Fix TestReadRetries Sub-task Resolved Shashikant Banerjee   Actions
        24.
        Fix TestOzoneManagerHA#testIncrementalWaitTimeWithSameNodeFailover Sub-task Resolved Unassigned   Actions
        25.
        Fix TestOzoneManagerHA#testMultipartUploadWithOneOmNodeDown Sub-task Resolved Unassigned   Actions
        26.
        Fix TestBlockOutputStreamWithFailures#test2DatanodesFailure Sub-task Resolved Shashikant Banerjee   Actions
        27.
        Fix TestOzoneRpcClientAbstract#testDeletedKeyForGDPR Sub-task Resolved Lokesh Jain   Actions

        Activity

          This comment will be Viewable by All Users Viewable by All Users
          Cancel

          People

            dteng Dave Teng
            elek Marton Elek
            Votes:
            0 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Time Tracking

                Estimated:
                Original Estimate - Not Specified
                Not Specified
                Remaining:
                Remaining Estimate - 0h
                0h
                Logged:
                Time Spent - 5h
                5h

                Slack

                  Issue deployment