Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: HDFS-5535 (Rolling upgrades)
    • Component/s: test
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      TestRollingUpgrade#testSecondaryNameNode causes subsequent tests to fail. It seems to be caused by the terminate hook used by the test. Commenting out this test case makes other tests in the same class pass.

      1. h5963_20140218.patch
        20 kB
        Tsz Wo Nicholas Sze
      2. h5963_20140219.patch
        20 kB
        Tsz Wo Nicholas Sze

        Activity

        Hide
        Tsz Wo Nicholas Sze added a comment -

        Since the edit log upgrade marker is changed to OP_ROLLING_UPGRADE_[START/FINALIZE], we could

        • remove RollingUpgradeStartupOption.STARTED;
        • split FSNamesystem.[start/finalize]RollingUpgrade to two methods for rpc and editlog processing; and
        • combine TestEditLogUpgradeMarker with TestRollingUpgrade.

        h5963_20140218.patch: moves testSecondaryNameNode to the end; fixes QJournalProtocolServerSideTranslatorPB and FileJournalManager; and also has the above changes.

        Show
        Tsz Wo Nicholas Sze added a comment - Since the edit log upgrade marker is changed to OP_ROLLING_UPGRADE_[START/FINALIZE], we could remove RollingUpgradeStartupOption.STARTED; split FSNamesystem.[start/finalize]RollingUpgrade to two methods for rpc and editlog processing; and combine TestEditLogUpgradeMarker with TestRollingUpgrade. h5963_20140218.patch: moves testSecondaryNameNode to the end; fixes QJournalProtocolServerSideTranslatorPB and FileJournalManager; and also has the above changes.
        Hide
        Arpit Agarwal added a comment -

        Thanks Nicholas. JDK7 could randomize the test case order so perhaps we need to put testSecondaryNameNode in a separate test class?

        Is this failure expected?

        Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 30.565 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestRollingUpgrade
        testRollback(org.apache.hadoop.hdfs.TestRollingUpgrade)  Time elapsed: 3.386 sec  <<< ERROR!
        java.io.IOException: There appears to be a gap in the edit log.  We expected txid 5, but got txid 8.
                at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94)
                at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:203)
                at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:131)
        
        Show
        Arpit Agarwal added a comment - Thanks Nicholas. JDK7 could randomize the test case order so perhaps we need to put testSecondaryNameNode in a separate test class? Is this failure expected? Tests run: 5, Failures: 0, Errors: 1, Skipped: 0, Time elapsed: 30.565 sec <<< FAILURE! - in org.apache.hadoop.hdfs.TestRollingUpgrade testRollback(org.apache.hadoop.hdfs.TestRollingUpgrade) Time elapsed: 3.386 sec <<< ERROR! java.io.IOException: There appears to be a gap in the edit log. We expected txid 5, but got txid 8. at org.apache.hadoop.hdfs.server.namenode.MetaRecoveryContext.editLogLoaderPrompt(MetaRecoveryContext.java:94) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadEditRecords(FSEditLogLoader.java:203) at org.apache.hadoop.hdfs.server.namenode.FSEditLogLoader.loadFSEdits(FSEditLogLoader.java:131)
        Hide
        Tsz Wo Nicholas Sze added a comment -

        For testSecondaryNameNode, let's simply remove it since it is not very useful.

        Let me also fix the bug in rollback.

        Show
        Tsz Wo Nicholas Sze added a comment - For testSecondaryNameNode, let's simply remove it since it is not very useful. Let me also fix the bug in rollback.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        > Let me also fix the bug in rollback.

        Talk to Jing Zhao, the rollback bug is quite involved so that we will fix it separately.

        h5963_20140219.patch: removes testSecondaryNameNode() and comments out restartNameNode() in testRollback().

        Show
        Tsz Wo Nicholas Sze added a comment - > Let me also fix the bug in rollback. Talk to Jing Zhao , the rollback bug is quite involved so that we will fix it separately. h5963_20140219.patch: removes testSecondaryNameNode() and comments out restartNameNode() in testRollback().
        Hide
        Arpit Agarwal added a comment -

        +1 for the patch. I committed it to the feature branch. Thanks Nicholas.

        Show
        Arpit Agarwal added a comment - +1 for the patch. I committed it to the feature branch. Thanks Nicholas.

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Arpit Agarwal
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development