Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17290

Potential loss of data for replication of bulk loaded hfiles

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.3.0
    • 1.4.0, 2.0.0
    • None
    • Reviewed

    Description

      Currently the support for replication of bulk loaded hfiles relies on bulk load marker written in the WAL.

      The move of bulk loaded hfile(s) (into region directory) may succeed but the write of bulk load marker may fail.
      This means that although bulk loaded hfile is being served in source cluster, the replication wouldn't happen.

      Normally operator is supposed to retry the bulk load. But relying on human retry is not robust solution.

      Attachments

        1. HBASE-17290.patch
          24 kB
          Ashish Singhi
        2. HBASE-17290.v1.patch
          27 kB
          Ashish Singhi
        3. HBASE-17290.branch-1.patch
          25 kB
          Ashish Singhi

        Issue Links

          Activity

            anoop.hbase Anoop Sam John added a comment -

            I believe this point was discussed in the original feature issue.. Sorry I totally forgot abt it now.. ashish singhi on vacation right now and once he is back, he can comment abt the discussion points happened at that time.

            anoop.hbase Anoop Sam John added a comment - I believe this point was discussed in the original feature issue.. Sorry I totally forgot abt it now.. ashish singhi on vacation right now and once he is back, he can comment abt the discussion points happened at that time.
            ashish singhi Ashish Singhi added a comment -

            tedyu, isn't this already handled as part of HBASE-15425 ?

            ashish singhi Ashish Singhi added a comment - tedyu , isn't this already handled as part of HBASE-15425 ?
            yuzhihong@gmail.com Ted Yu added a comment - - edited

            HBASE-15446 is still open.

            As the description of HBASE-15446 says, the solution depends on client to retry. What if client doesn't retry ?
            See the latest patch on HBASE-14417 for solution - we record the hfile before the movement into region directory using the hook introduced by HBASE-17292.

            yuzhihong@gmail.com Ted Yu added a comment - - edited HBASE-15446 is still open. As the description of HBASE-15446 says, the solution depends on client to retry. What if client doesn't retry ? See the latest patch on HBASE-14417 for solution - we record the hfile before the movement into region directory using the hook introduced by HBASE-17292 .
            yuzhihong@gmail.com Ted Yu added a comment - - edited

            The latest patch for HBASE-14417 is on reviewboard and attached to JIRA.

            yuzhihong@gmail.com Ted Yu added a comment - - edited The latest patch for HBASE-14417 is on reviewboard and attached to JIRA.
            yuzhihong@gmail.com Ted Yu added a comment -

            ashish singhi:
            What do you think ?

            yuzhihong@gmail.com Ted Yu added a comment - ashish singhi : What do you think ?
            ashish singhi Ashish Singhi added a comment - - edited

            Sorry for the delay, got stuck with company work.
            I have attached the patch.

            Added a new RS observer, ReplicationObserver to solve this bug.
            Please review.

            ashish singhi Ashish Singhi added a comment - - edited Sorry for the delay, got stuck with company work. I have attached the patch. Added a new RS observer, ReplicationObserver to solve this bug. Please review.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 13s Docker mode activated.
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
            0 mvndep 0m 23s Maven dependency ordering for branch
            +1 mvninstall 3m 6s master passed
            +1 compile 0m 49s master passed
            +1 checkstyle 0m 42s master passed
            +1 mvneclipse 0m 23s master passed
            +1 findbugs 2m 29s master passed
            +1 javadoc 0m 43s master passed
            0 mvndep 0m 11s Maven dependency ordering for patch
            +1 mvninstall 0m 56s the patch passed
            +1 compile 0m 50s the patch passed
            +1 javac 0m 50s the patch passed
            +1 checkstyle 0m 42s the patch passed
            +1 mvneclipse 0m 22s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 hadoopcheck 26m 35s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1.
            +1 findbugs 2m 46s the patch passed
            +1 javadoc 0m 43s the patch passed
            +1 unit 1m 5s hbase-client in the patch passed.
            -1 unit 81m 7s hbase-server in the patch failed.
            +1 asflicense 0m 29s The patch does not generate ASF License warnings.
            125m 5s



            Reason Tests
            Failed junit tests hadoop.hbase.replication.TestMasterReplication
              hadoop.hbase.regionserver.TestHRegion



            Subsystem Report/Notes
            Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845785/HBASE-17290.patch
            JIRA Issue HBASE-17290
            Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
            uname Linux ef0fbfb7688c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision master / dba103e
            Default Java 1.8.0_111
            findbugs v3.0.0
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/5145/artifact/patchprocess/patch-unit-hbase-server.txt
            unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/5145/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/5145/testReport/
            modules C: hbase-client hbase-server U: .
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/5145/console
            Powered by Apache Yetus 0.3.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 23s Maven dependency ordering for branch +1 mvninstall 3m 6s master passed +1 compile 0m 49s master passed +1 checkstyle 0m 42s master passed +1 mvneclipse 0m 23s master passed +1 findbugs 2m 29s master passed +1 javadoc 0m 43s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 56s the patch passed +1 compile 0m 50s the patch passed +1 javac 0m 50s the patch passed +1 checkstyle 0m 42s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 26m 35s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. +1 findbugs 2m 46s the patch passed +1 javadoc 0m 43s the patch passed +1 unit 1m 5s hbase-client in the patch passed. -1 unit 81m 7s hbase-server in the patch failed. +1 asflicense 0m 29s The patch does not generate ASF License warnings. 125m 5s Reason Tests Failed junit tests hadoop.hbase.replication.TestMasterReplication   hadoop.hbase.regionserver.TestHRegion Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845785/HBASE-17290.patch JIRA Issue HBASE-17290 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux ef0fbfb7688c 3.13.0-95-generic #142-Ubuntu SMP Fri Aug 12 17:00:09 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / dba103e Default Java 1.8.0_111 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HBASE-Build/5145/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/5145/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/5145/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/5145/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
            yuzhihong@gmail.com Ted Yu added a comment -

            Test failure was caused by:

            Caused by: java.lang.NullPointerException
            	at org.apache.hadoop.hbase.replication.TestMasterReplication.startMiniClusters(TestMasterReplication.java:492)
            	at org.apache.hadoop.hbase.replication.TestMasterReplication.testHFileMultiSlaveReplication(TestMasterReplication.java:331)
            

            Please fix in the next patch.

            yuzhihong@gmail.com Ted Yu added a comment - Test failure was caused by: Caused by: java.lang.NullPointerException at org.apache.hadoop.hbase.replication.TestMasterReplication.startMiniClusters(TestMasterReplication.java:492) at org.apache.hadoop.hbase.replication.TestMasterReplication.testHFileMultiSlaveReplication(TestMasterReplication.java:331) Please fix in the next patch.
            yuzhihong@gmail.com Ted Yu added a comment -

            Please enrich the javadoc with explanation for the second component of the Pair.

            +   * @param pairs list of hfile references to be added
            

            Please make the parameter name consistent - should be pairs

            +  public void addHFileRefs(String peerId, List<Pair<Path, Path>> files)
            

            For ReplicationObserver.java, please reference the format of license header of existing classes.
            Consider adding @InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.CONFIG) to this class.

            +      LOG.debug("Skipping recording bulk load entries in preCommitStoreFile for bulkloaded "
            +          + "data replication.");
            

            It would be better if the case for bulk load replication and the case where pairs is empty are logged separately.
            This would facilitate troubleshooting.

            Can you point me to the code which handles the case where Path for bulk loaded hfile is recorded but the commit (move of hfile) fails ?
            In that scenario, the file wouldn't be found at time of replication.

            yuzhihong@gmail.com Ted Yu added a comment - Please enrich the javadoc with explanation for the second component of the Pair. + * @param pairs list of hfile references to be added Please make the parameter name consistent - should be pairs + public void addHFileRefs( String peerId, List<Pair<Path, Path>> files) For ReplicationObserver.java, please reference the format of license header of existing classes. Consider adding @InterfaceAudience.LimitedPrivate(HBaseInterfaceAudience.CONFIG) to this class. + LOG.debug( "Skipping recording bulk load entries in preCommitStoreFile for bulkloaded " + + "data replication." ); It would be better if the case for bulk load replication and the case where pairs is empty are logged separately. This would facilitate troubleshooting. Can you point me to the code which handles the case where Path for bulk loaded hfile is recorded but the commit (move of hfile) fails ? In that scenario, the file wouldn't be found at time of replication.
            ashish singhi Ashish Singhi added a comment -

            Can you point me to the code which handles the case where Path for bulk loaded hfile is recorded but the commit (move of hfile) fails ?
            In that scenario, the file wouldn't be found at time of replication.

            In that scenario we will end up here, https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java#L380

            ashish singhi Ashish Singhi added a comment - Can you point me to the code which handles the case where Path for bulk loaded hfile is recorded but the commit (move of hfile) fails ? In that scenario, the file wouldn't be found at time of replication. In that scenario we will end up here, https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java#L380
            yuzhihong@gmail.com Ted Yu added a comment -

            The LOG in that catch block is at error level.
            Consider changing it to DEBUG since the absence of hfile implies error in the commit step (which should be remedied by operator retry).

            yuzhihong@gmail.com Ted Yu added a comment - The LOG in that catch block is at error level. Consider changing it to DEBUG since the absence of hfile implies error in the commit step (which should be remedied by operator retry).
            ashish singhi Ashish Singhi added a comment -

            Addressed the comments.
            Please review.

            ashish singhi Ashish Singhi added a comment - Addressed the comments. Please review.
            yuzhihong@gmail.com Ted Yu added a comment -

            Latest patch looks good - assuming TestMasterReplication passes.

            yuzhihong@gmail.com Ted Yu added a comment - Latest patch looks good - assuming TestMasterReplication passes.
            hadoopqa Hadoop QA added a comment -
            +1 overall



            Vote Subsystem Runtime Comment
            0 reexec 0m 10s Docker mode activated.
            +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
            0 mvndep 0m 10s Maven dependency ordering for branch
            +1 mvninstall 2m 51s master passed
            +1 compile 0m 50s master passed
            +1 checkstyle 0m 42s master passed
            +1 mvneclipse 0m 21s master passed
            +1 findbugs 2m 29s master passed
            +1 javadoc 0m 43s master passed
            0 mvndep 0m 11s Maven dependency ordering for patch
            +1 mvninstall 0m 57s the patch passed
            +1 compile 0m 51s the patch passed
            +1 javac 0m 51s the patch passed
            +1 checkstyle 0m 42s the patch passed
            +1 mvneclipse 0m 22s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 hadoopcheck 26m 4s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1.
            +1 findbugs 2m 47s the patch passed
            +1 javadoc 0m 43s the patch passed
            +1 unit 1m 1s hbase-client in the patch passed.
            +1 unit 80m 47s hbase-server in the patch passed.
            +1 asflicense 0m 26s The patch does not generate ASF License warnings.
            123m 36s



            Subsystem Report/Notes
            Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845828/HBASE-17290.v1.patch
            JIRA Issue HBASE-17290
            Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
            uname Linux 45b15a51594d 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
            git revision master / df98d8d
            Default Java 1.8.0_111
            findbugs v3.0.0
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/5149/testReport/
            modules C: hbase-client hbase-server U: .
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/5149/console
            Powered by Apache Yetus 0.3.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 10s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 2m 51s master passed +1 compile 0m 50s master passed +1 checkstyle 0m 42s master passed +1 mvneclipse 0m 21s master passed +1 findbugs 2m 29s master passed +1 javadoc 0m 43s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 57s the patch passed +1 compile 0m 51s the patch passed +1 javac 0m 51s the patch passed +1 checkstyle 0m 42s the patch passed +1 mvneclipse 0m 22s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 26m 4s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha1. +1 findbugs 2m 47s the patch passed +1 javadoc 0m 43s the patch passed +1 unit 1m 1s hbase-client in the patch passed. +1 unit 80m 47s hbase-server in the patch passed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 123m 36s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845828/HBASE-17290.v1.patch JIRA Issue HBASE-17290 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 45b15a51594d 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / df98d8d Default Java 1.8.0_111 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/5149/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/5149/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
            ashish singhi Ashish Singhi added a comment -

            Planning to commit this after an hour or so, if there is no further review comments.

            ashish singhi Ashish Singhi added a comment - Planning to commit this after an hour or so, if there is no further review comments.
            ashish singhi Ashish Singhi added a comment -

            Attached branch-1 patch.

            ashish singhi Ashish Singhi added a comment - Attached branch-1 patch.
            hadoopqa Hadoop QA added a comment -
            -1 overall



            Vote Subsystem Runtime Comment
            0 reexec 10m 37s Docker mode activated.
            +1 @author 0m 0s The patch does not contain any @author tags.
            +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
            0 mvndep 0m 21s Maven dependency ordering for branch
            +1 mvninstall 1m 42s branch-1 passed
            +1 compile 0m 44s branch-1 passed with JDK v1.8.0_111
            +1 compile 0m 50s branch-1 passed with JDK v1.7.0_80
            +1 checkstyle 0m 52s branch-1 passed
            +1 mvneclipse 0m 28s branch-1 passed
            -1 findbugs 1m 54s hbase-server in branch-1 has 2 extant Findbugs warnings.
            +1 javadoc 0m 39s branch-1 passed with JDK v1.8.0_111
            +1 javadoc 0m 52s branch-1 passed with JDK v1.7.0_80
            0 mvndep 0m 12s Maven dependency ordering for patch
            +1 mvninstall 0m 59s the patch passed
            +1 compile 0m 44s the patch passed with JDK v1.8.0_111
            +1 javac 0m 44s the patch passed
            +1 compile 0m 51s the patch passed with JDK v1.7.0_80
            -1 javac 0m 34s hbase-server-jdk1.7.0_80 with JDK v1.7.0_80 generated 1 new + 4 unchanged - 1 fixed = 5 total (was 5)
            +1 checkstyle 0m 52s the patch passed
            +1 mvneclipse 0m 27s the patch passed
            +1 whitespace 0m 0s The patch has no whitespace issues.
            +1 hadoopcheck 14m 57s The patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1.
            +1 hbaseprotoc 0m 25s the patch passed
            +1 findbugs 3m 20s the patch passed
            +1 javadoc 0m 39s the patch passed with JDK v1.8.0_111
            +1 javadoc 0m 53s the patch passed with JDK v1.7.0_80
            +1 unit 1m 50s hbase-client in the patch passed.
            -1 unit 84m 50s hbase-server in the patch failed.
            +1 asflicense 0m 32s The patch does not generate ASF License warnings.
            132m 14s



            Reason Tests
            Failed junit tests hadoop.hbase.client.TestMetaWithReplicas



            Subsystem Report/Notes
            Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:e01ee2f
            JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845959/HBASE-17290.branch-1.patch
            JIRA Issue HBASE-17290
            Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
            uname Linux 9dd280c94211 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
            Build tool maven
            Personality /testptch/patchprocess/precommit/personality/hbase.sh
            git revision branch-1 / 667c5eb
            Default Java 1.7.0_80
            Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-oracle:1.7.0_80
            findbugs v3.0.0
            findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
            javac https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/diff-compile-javac-hbase-server-jdk1.7.0_80.txt
            unit https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/patch-unit-hbase-server.txt
            unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/patch-unit-hbase-server.txt
            Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/5158/testReport/
            modules C: hbase-client hbase-server U: .
            Console output https://builds.apache.org/job/PreCommit-HBASE-Build/5158/console
            Powered by Apache Yetus 0.3.0 http://yetus.apache.org

            This message was automatically generated.

            hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 10m 37s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 21s Maven dependency ordering for branch +1 mvninstall 1m 42s branch-1 passed +1 compile 0m 44s branch-1 passed with JDK v1.8.0_111 +1 compile 0m 50s branch-1 passed with JDK v1.7.0_80 +1 checkstyle 0m 52s branch-1 passed +1 mvneclipse 0m 28s branch-1 passed -1 findbugs 1m 54s hbase-server in branch-1 has 2 extant Findbugs warnings. +1 javadoc 0m 39s branch-1 passed with JDK v1.8.0_111 +1 javadoc 0m 52s branch-1 passed with JDK v1.7.0_80 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 0m 59s the patch passed +1 compile 0m 44s the patch passed with JDK v1.8.0_111 +1 javac 0m 44s the patch passed +1 compile 0m 51s the patch passed with JDK v1.7.0_80 -1 javac 0m 34s hbase-server-jdk1.7.0_80 with JDK v1.7.0_80 generated 1 new + 4 unchanged - 1 fixed = 5 total (was 5) +1 checkstyle 0m 52s the patch passed +1 mvneclipse 0m 27s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 14m 57s The patch does not cause any errors with Hadoop 2.4.0 2.4.1 2.5.0 2.5.1 2.5.2 2.6.1 2.6.2 2.6.3 2.7.1. +1 hbaseprotoc 0m 25s the patch passed +1 findbugs 3m 20s the patch passed +1 javadoc 0m 39s the patch passed with JDK v1.8.0_111 +1 javadoc 0m 53s the patch passed with JDK v1.7.0_80 +1 unit 1m 50s hbase-client in the patch passed. -1 unit 84m 50s hbase-server in the patch failed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 132m 14s Reason Tests Failed junit tests hadoop.hbase.client.TestMetaWithReplicas Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:e01ee2f JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12845959/HBASE-17290.branch-1.patch JIRA Issue HBASE-17290 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 9dd280c94211 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/patchprocess/precommit/personality/hbase.sh git revision branch-1 / 667c5eb Default Java 1.7.0_80 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_111 /usr/lib/jvm/java-7-oracle:1.7.0_80 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javac https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/diff-compile-javac-hbase-server-jdk1.7.0_80.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/5158/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/5158/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/5158/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
            ashish singhi Ashish Singhi added a comment -

            Failed junit tests hadoop.hbase.client.TestMetaWithReplicas

            Test failure was not related to patch. It is failing on other build runs also here, here...

            ashish singhi Ashish Singhi added a comment - Failed junit tests hadoop.hbase.client.TestMetaWithReplicas Test failure was not related to patch. It is failing on other build runs also here , here ...
            ashish singhi Ashish Singhi added a comment -

            Pushed to master and branch-1.
            Thanks for the reviews, Ted.

            ashish singhi Ashish Singhi added a comment - Pushed to master and branch-1. Thanks for the reviews, Ted.
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-1.4 #585 (See https://builds.apache.org/job/HBase-1.4/585/)
            HBASE-17290 Potential loss of data for replication of bulk loaded hfiles (ashishsinghi: rev e8e40d86258a2f19838f3aea55dafbbc1b860942)

            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            • (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-1.4 #585 (See https://builds.apache.org/job/HBase-1.4/585/ ) HBASE-17290 Potential loss of data for replication of bulk loaded hfiles (ashishsinghi: rev e8e40d86258a2f19838f3aea55dafbbc1b860942) (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2269 (See https://builds.apache.org/job/HBase-Trunk_matrix/2269/)
            HBASE-17290 Potential loss of data for replication of bulk loaded hfiles (ashishsinghi: rev 5f631b9653a4bf86a2bebed58abed747c04b704f)

            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/TableBasedReplicationQueuesImpl.java
            • (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-Trunk_matrix #2269 (See https://builds.apache.org/job/HBase-Trunk_matrix/2269/ ) HBASE-17290 Potential loss of data for replication of bulk loaded hfiles (ashishsinghi: rev 5f631b9653a4bf86a2bebed58abed747c04b704f) (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/TableBasedReplicationQueuesImpl.java (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #291 (See https://builds.apache.org/job/HBase-1.3-JDK7/291/)
            HBASE-18900 Backport HBASE-17290 "Potential loss of data for replication (tedyu: rev 8e5f5dd655d6f16c0d9f421c308a05ce5fa61c72)

            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #291 (See https://builds.apache.org/job/HBase-1.3-JDK7/291/ ) HBASE-18900 Backport HBASE-17290 "Potential loss of data for replication (tedyu: rev 8e5f5dd655d6f16c0d9f421c308a05ce5fa61c72) (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build HBase-1.3-IT #219 (See https://builds.apache.org/job/HBase-1.3-IT/219/)
            HBASE-18900 Backport HBASE-17290 "Potential loss of data for replication (tedyu: rev 8e5f5dd655d6f16c0d9f421c308a05ce5fa61c72)

            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            • (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-1.3-IT #219 (See https://builds.apache.org/job/HBase-1.3-IT/219/ ) HBASE-18900 Backport HBASE-17290 "Potential loss of data for replication (tedyu: rev 8e5f5dd655d6f16c0d9f421c308a05ce5fa61c72) (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #303 (See https://builds.apache.org/job/HBase-1.3-JDK8/303/)
            HBASE-18900 Backport HBASE-17290 "Potential loss of data for replication (tedyu: rev 8e5f5dd655d6f16c0d9f421c308a05ce5fa61c72)

            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-1.3-JDK8 #303 (See https://builds.apache.org/job/HBase-1.3-JDK8/303/ ) HBASE-18900 Backport HBASE-17290 "Potential loss of data for replication (tedyu: rev 8e5f5dd655d6f16c0d9f421c308a05ce5fa61c72) (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (add) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            hudson Hudson added a comment -

            SUCCESS: Integrated in Jenkins build HBase-1.3-IT #230 (See https://builds.apache.org/job/HBase-1.3-IT/230/)
            HBASE-18900 Backport HBASE-17290 - revert due to incompatibility (tedyu: rev ccf9b85d040e9028bcab0d49321be104d67fc76e)

            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            hudson Hudson added a comment - SUCCESS: Integrated in Jenkins build HBase-1.3-IT #230 (See https://builds.apache.org/job/HBase-1.3-IT/230/ ) HBASE-18900 Backport HBASE-17290 - revert due to incompatibility (tedyu: rev ccf9b85d040e9028bcab0d49321be104d67fc76e) (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #302 (See https://builds.apache.org/job/HBase-1.3-JDK7/302/)
            HBASE-18900 Backport HBASE-17290 - revert due to incompatibility (tedyu: rev ccf9b85d040e9028bcab0d49321be104d67fc76e)

            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            • (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-1.3-JDK7 #302 (See https://builds.apache.org/job/HBase-1.3-JDK7/302/ ) HBASE-18900 Backport HBASE-17290 - revert due to incompatibility (tedyu: rev ccf9b85d040e9028bcab0d49321be104d67fc76e) (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            hudson Hudson added a comment -

            FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #316 (See https://builds.apache.org/job/HBase-1.3-JDK8/316/)
            HBASE-18900 Backport HBASE-17290 - revert due to incompatibility (tedyu: rev ccf9b85d040e9028bcab0d49321be104d67fc76e)

            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java
            • (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java
            • (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java
            • (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java
            • (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            hudson Hudson added a comment - FAILURE: Integrated in Jenkins build HBase-1.3-JDK8 #316 (See https://builds.apache.org/job/HBase-1.3-JDK8/316/ ) HBASE-18900 Backport HBASE-17290 - revert due to incompatibility (tedyu: rev ccf9b85d040e9028bcab0d49321be104d67fc76e) (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueues.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceManager.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSource.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/master/cleaner/TestReplicationHFileCleaner.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationSourceInterface.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/ReplicationSourceDummy.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/HRegionServer.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/HFileReplicator.java (edit) hbase-client/src/main/java/org/apache/hadoop/hbase/replication/ReplicationQueuesZKImpl.java (edit) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/Replication.java (edit) hbase-server/src/test/java/org/apache/hadoop/hbase/replication/TestReplicationStateBasic.java (delete) hbase-server/src/main/java/org/apache/hadoop/hbase/replication/regionserver/ReplicationObserver.java
            anoop.hbase Anoop Sam John added a comment -

            Sorry for asking it too late.
            When write marker cell in WAL is failed, do we succed the bulk load op still?
            On the new CP, we use post commit hook of store file. So this will be called for all files including the flushed ones right. So when this CP is in place, how will we treat diff btw bulk loaded files and normal flushed/compaction result files?
            The CP seems to listen for the post commit hook and then add an entry into the hfiles for replication Q. I wonder why we cannot do this in core code flow itself. WHy we need an extra CP? This is not a sync HFile replication op. Just adding an entry. Said that even the WAL write also can be avoided?
            Really sorry for asking this late after its commit. Got to see this code while doing the recent CP cleanups only.

            anoop.hbase Anoop Sam John added a comment - Sorry for asking it too late. When write marker cell in WAL is failed, do we succed the bulk load op still? On the new CP, we use post commit hook of store file. So this will be called for all files including the flushed ones right. So when this CP is in place, how will we treat diff btw bulk loaded files and normal flushed/compaction result files? The CP seems to listen for the post commit hook and then add an entry into the hfiles for replication Q. I wonder why we cannot do this in core code flow itself. WHy we need an extra CP? This is not a sync HFile replication op. Just adding an entry. Said that even the WAL write also can be avoided? Really sorry for asking this late after its commit. Got to see this code while doing the recent CP cleanups only.

            People

              ashish singhi Ashish Singhi
              yuzhihong@gmail.com Ted Yu
              Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: