Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha2
    • Component/s: ha
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      The BookkeeperJournalManager implementation has been removed. Users are encouraged to use QuorumJournalManager instead.

      Description

      Based on discussions done in mailing list here BKJM is no longer required as QJM supports the required functionality with minimum deployment overhead.

      This Jira is for tracking purpose.

      1. HDFS-10957-01.patch
        178 kB
        Vinayakumar B
      2. HDFS-10957-02.patch
        181 kB
        Vinayakumar B

        Activity

        Hide
        vinayrpet Vinayakumar B added a comment -

        Attaching patch for jenkins

        Show
        vinayrpet Vinayakumar B added a comment - Attaching patch for jenkins
        Hide
        hadoopqa Hadoop QA added a comment -
        +1 overall



        Vote Subsystem Runtime Comment
        0 reexec 19m 22s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 24s Maven dependency ordering for branch
        +1 mvninstall 8m 45s trunk passed
        +1 compile 0m 52s trunk passed
        +1 checkstyle 0m 27s trunk passed
        +1 mvnsite 1m 21s trunk passed
        +1 mvneclipse 0m 24s trunk passed
        +1 findbugs 2m 21s trunk passed
        +1 javadoc 1m 16s trunk passed
        +1 mvninstall 0m 52s the patch passed
        +1 compile 0m 49s the patch passed
        +1 cc 0m 49s the patch passed
        +1 javac 0m 49s the patch passed
        +1 checkstyle 0m 26s the patch passed
        +1 mvnsite 0m 56s the patch passed
        +1 mvneclipse 0m 10s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 0s The patch has no ill-formed XML file.
        +1 findbugs 1m 58s the patch passed
        +1 javadoc 0m 59s the patch passed
        +1 unit 59m 44s hadoop-hdfs in the patch passed.
        0 asflicense 0m 8s ASF License check generated no output?
        103m 7s



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10957
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831564/HDFS-10957-01.patch
        Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 4fc607249805 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 382307c
        Default Java 1.8.0_101
        findbugs v3.0.0
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17002/testReport/
        modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17002/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 19m 22s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 24s Maven dependency ordering for branch +1 mvninstall 8m 45s trunk passed +1 compile 0m 52s trunk passed +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 21s trunk passed +1 mvneclipse 0m 24s trunk passed +1 findbugs 2m 21s trunk passed +1 javadoc 1m 16s trunk passed +1 mvninstall 0m 52s the patch passed +1 compile 0m 49s the patch passed +1 cc 0m 49s the patch passed +1 javac 0m 49s the patch passed +1 checkstyle 0m 26s the patch passed +1 mvnsite 0m 56s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 0s The patch has no ill-formed XML file. +1 findbugs 1m 58s the patch passed +1 javadoc 0m 59s the patch passed +1 unit 59m 44s hadoop-hdfs in the patch passed. 0 asflicense 0m 8s ASF License check generated no output? 103m 7s Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10957 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831564/HDFS-10957-01.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 4fc607249805 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 382307c Default Java 1.8.0_101 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17002/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17002/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        kihwal Kihwal Lee added a comment -

        hadoop-hdfs-project/pom.xml needs to be updated.

        Show
        kihwal Kihwal Lee added a comment - hadoop-hdfs-project/pom.xml needs to be updated.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Thanks Kihwal Lee, I will update the patch.

        Show
        vinayrpet Vinayakumar B added a comment - Thanks Kihwal Lee , I will update the patch.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Updated the patch

        Show
        vinayrpet Vinayakumar B added a comment - Updated the patch
        Hide
        hadoopqa Hadoop QA added a comment -
        -1 overall



        Vote Subsystem Runtime Comment
        0 reexec 0m 15s Docker mode activated.
        +1 @author 0m 0s The patch does not contain any @author tags.
        +1 test4tests 0m 0s The patch appears to include 11 new or modified test files.
        0 mvndep 0m 15s Maven dependency ordering for branch
        +1 mvninstall 6m 54s trunk passed
        +1 compile 7m 26s trunk passed
        +1 checkstyle 1m 34s trunk passed
        +1 mvnsite 3m 8s trunk passed
        +1 mvneclipse 0m 54s trunk passed
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project
        +1 findbugs 2m 31s trunk passed
        +1 javadoc 3m 6s trunk passed
        0 mvndep 0m 16s Maven dependency ordering for patch
        +1 mvninstall 2m 51s the patch passed
        +1 compile 9m 33s the patch passed
        +1 cc 9m 33s the patch passed
        +1 javac 9m 33s root generated 0 new + 707 unchanged - 1 fixed = 707 total (was 708)
        +1 checkstyle 1m 31s root: The patch generated 0 new + 0 unchanged - 138 fixed = 0 total (was 138)
        +1 mvnsite 2m 42s the patch passed
        +1 mvneclipse 0m 43s the patch passed
        +1 whitespace 0m 0s The patch has no whitespace issues.
        +1 xml 0m 2s The patch has no ill-formed XML file.
        0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project
        +1 findbugs 1m 52s the patch passed
        +1 javadoc 0m 12s hadoop-project in the patch passed.
        +1 javadoc 0m 59s hadoop-hdfs in the patch passed.
        +1 javadoc 1m 22s hadoop-hdfs-project generated 0 new + 65 unchanged - 61 fixed = 65 total (was 126)
        +1 unit 0m 11s hadoop-project in the patch passed.
        -1 unit 76m 5s hadoop-hdfs in the patch failed.
        -1 unit 78m 11s hadoop-hdfs-project in the patch failed.
        +1 asflicense 0m 24s The patch does not generate ASF License warnings.
        204m 23s



        Reason Tests
        Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency
          hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency



        Subsystem Report/Notes
        Docker Image:yetus/hadoop:9560f25
        JIRA Issue HDFS-10957
        JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831791/HDFS-10957-02.patch
        Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc
        uname Linux 8d336dbc37b3 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
        Build tool maven
        Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
        git revision trunk / 8867762
        Default Java 1.8.0_101
        findbugs v3.0.0
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17023/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt
        unit https://builds.apache.org/job/PreCommit-HDFS-Build/17023/artifact/patchprocess/patch-unit-hadoop-hdfs-project.txt
        Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17023/testReport/
        modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project U: .
        Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17023/console
        Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

        This message was automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 11 new or modified test files. 0 mvndep 0m 15s Maven dependency ordering for branch +1 mvninstall 6m 54s trunk passed +1 compile 7m 26s trunk passed +1 checkstyle 1m 34s trunk passed +1 mvnsite 3m 8s trunk passed +1 mvneclipse 0m 54s trunk passed 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project +1 findbugs 2m 31s trunk passed +1 javadoc 3m 6s trunk passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 2m 51s the patch passed +1 compile 9m 33s the patch passed +1 cc 9m 33s the patch passed +1 javac 9m 33s root generated 0 new + 707 unchanged - 1 fixed = 707 total (was 708) +1 checkstyle 1m 31s root: The patch generated 0 new + 0 unchanged - 138 fixed = 0 total (was 138) +1 mvnsite 2m 42s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 xml 0m 2s The patch has no ill-formed XML file. 0 findbugs 0m 0s Skipped patched modules with no Java source: hadoop-project hadoop-hdfs-project +1 findbugs 1m 52s the patch passed +1 javadoc 0m 12s hadoop-project in the patch passed. +1 javadoc 0m 59s hadoop-hdfs in the patch passed. +1 javadoc 1m 22s hadoop-hdfs-project generated 0 new + 65 unchanged - 61 fixed = 65 total (was 126) +1 unit 0m 11s hadoop-project in the patch passed. -1 unit 76m 5s hadoop-hdfs in the patch failed. -1 unit 78m 11s hadoop-hdfs-project in the patch failed. +1 asflicense 0m 24s The patch does not generate ASF License warnings. 204m 23s Reason Tests Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency   hadoop.hdfs.server.namenode.TestNameNodeMetadataConsistency Subsystem Report/Notes Docker Image:yetus/hadoop:9560f25 JIRA Issue HDFS-10957 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12831791/HDFS-10957-02.patch Optional Tests asflicense xml compile javac javadoc mvninstall mvnsite unit findbugs checkstyle cc uname Linux 8d336dbc37b3 3.13.0-96-generic #143-Ubuntu SMP Mon Aug 29 20:15:20 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 8867762 Default Java 1.8.0_101 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/17023/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/17023/artifact/patchprocess/patch-unit-hadoop-hdfs-project.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/17023/testReport/ modules C: hadoop-project hadoop-hdfs-project/hadoop-hdfs hadoop-hdfs-project U: . Console output https://builds.apache.org/job/PreCommit-HDFS-Build/17023/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
        Hide
        brahmareddy Brahma Reddy Battula added a comment -

        Latest patch looks good to me , +1..Thanks Vinayakumar B

        Show
        brahmareddy Brahma Reddy Battula added a comment - Latest patch looks good to me , +1..Thanks Vinayakumar B
        Hide
        vinayrpet Vinayakumar B added a comment -

        Committed to trunk.
        Thanks Brahma Reddy Battula and Kihwal Lee
        And everyone replied in mailing thread.

        Show
        vinayrpet Vinayakumar B added a comment - Committed to trunk. Thanks Brahma Reddy Battula and Kihwal Lee And everyone replied in mailing thread.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10555 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10555/)
        HDFS-10957. Retire BKJM from trunk (Vinayakumar B) (vinayakumarb: rev 311954883f714973784432589896553eb320b597)

        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/CurrentInprogress.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources/log4j.properties
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestCurrentInprogress.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBootstrapStandbyWithBKJM.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperSpeculativeRead.java
        • (edit) hadoop-project/pom.xml
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperAsHASharedDir.java
        • (edit) hadoop-hdfs-project/pom.xml
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java
        • (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithNFS.md
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogOutputStream.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/BKJMUtil.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperEditLogStreams.java
        • (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build Hadoop-trunk-Commit #10555 (See https://builds.apache.org/job/Hadoop-trunk-Commit/10555/ ) HDFS-10957 . Retire BKJM from trunk (Vinayakumar B) (vinayakumarb: rev 311954883f714973784432589896553eb320b597) (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/CurrentInprogress.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/resources/log4j.properties (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperJournalManager.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/MaxTxId.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestCurrentInprogress.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/EditLogLedgerMetadata.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBootstrapStandbyWithBKJM.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperSpeculativeRead.java (edit) hadoop-project/pom.xml (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperAsHASharedDir.java (edit) hadoop-hdfs-project/pom.xml (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/dev-support/findbugsExcludeFile.xml (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperJournalManager.java (edit) hadoop-hdfs-project/hadoop-hdfs/src/site/markdown/HDFSHighAvailabilityWithNFS.md (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogTestUtil.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogOutputStream.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/BKJMUtil.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperHACheckpoints.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/pom.xml (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperConfiguration.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/java/org/apache/hadoop/contrib/bkjournal/BookKeeperEditLogInputStream.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/main/proto/bkjournal.proto (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/src/test/java/org/apache/hadoop/contrib/bkjournal/TestBookKeeperEditLogStreams.java (delete) hadoop-hdfs-project/hadoop-hdfs/src/contrib/bkjournal/README.txt
        Hide
        andrew.wang Andrew Wang added a comment -

        Looks like this was committed, so resolving the JIRA. I also added a release note, feel free to edit.

        Show
        andrew.wang Andrew Wang added a comment - Looks like this was committed, so resolving the JIRA. I also added a release note, feel free to edit.
        Hide
        vinayrpet Vinayakumar B added a comment -

        Oops. Forgot that.
        Thanks Andrew.

        Show
        vinayrpet Vinayakumar B added a comment - Oops. Forgot that. Thanks Andrew.

          People

          • Assignee:
            vinayrpet Vinayakumar B
            Reporter:
            vinayrpet Vinayakumar B
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development