Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17125

Inconsistent result when use filter to read data

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 3.0.0, 2.0.0-alpha-2
    • Component/s: None
    • Labels:
      None

      Description

      Assume a cloumn's max versions is 3, then we write 4 versions of this column. The oldest version doesn't remove immediately. But from the user view, the oldest version has gone. When user use a filter to query, if the filter skip a new version, then the oldest version will be seen again. But after compact the region, then the oldest version will never been seen. So it is weird for user. The query will get inconsistent result before and after region compaction.

      The reason is matchColumn method of UserScanQueryMatcher. It first check the cell by filter, then check the number of versions needed. So if the filter skip the new version, then the oldest version will be seen again when it is not removed.

      Have a discussion offline with Duo Zhang and Honghua Feng, now we have two solution for this problem. The first idea is check the number of versions first, then check the cell by filter. As the comment of setFilter, the filter is called after all tests for ttl, column match, deletes and max versions have been run.

        /**
         * Apply the specified server-side filter when performing the Query.
         * Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests
         * for ttl, column match, deletes and max versions have been run.
         * @param filter filter to run on the server
         * @return this for invocation chaining
         */
        public Query setFilter(Filter filter) {
          this.filter = filter;
          return this;
        }
      

      But this idea has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore.

      So the second idea has three steps.
      1. check by the max versions of this column
      2. check the kv by filter
      3. check the versions which user need.
      But this will lead the ScanQueryMatcher more complicated. And this will break the javadoc of Query.setFilter.

      Now we don't have a final solution for this problem. Suggestions are welcomed.

      1. 17125-slack-13.txt
        45 kB
        Ted Yu
      2. example.diff
        4 kB
        Guanghao Zhang
      3. HBASE-17125.master.001.patch
        12 kB
        Guanghao Zhang
      4. HBASE-17125.master.002.patch
        13 kB
        Guanghao Zhang
      5. HBASE-17125.master.002.patch
        13 kB
        Guanghao Zhang
      6. HBASE-17125.master.003.patch
        28 kB
        Guanghao Zhang
      7. HBASE-17125.master.004.patch
        29 kB
        Guanghao Zhang
      8. HBASE-17125.master.005.patch
        29 kB
        Guanghao Zhang
      9. HBASE-17125.master.006.patch
        30 kB
        Guanghao Zhang
      10. HBASE-17125.master.007.patch
        62 kB
        Guanghao Zhang
      11. HBASE-17125.master.008.patch
        62 kB
        Guanghao Zhang
      12. HBASE-17125.master.009.patch
        66 kB
        Guanghao Zhang
      13. HBASE-17125.master.009.patch
        65 kB
        Guanghao Zhang
      14. HBASE-17125.master.010.patch
        65 kB
        Guanghao Zhang
      15. HBASE-17125.master.011.patch
        65 kB
        Guanghao Zhang
      16. HBASE-17125.master.011.patch
        65 kB
        Guanghao Zhang
      17. HBASE-17125.master.012.patch
        51 kB
        Guanghao Zhang
      18. HBASE-17125.master.013.patch
        40 kB
        Guanghao Zhang
      19. HBASE-17125.master.014.patch
        33 kB
        Guanghao Zhang
      20. HBASE-17125.master.015.patch
        33 kB
        Guanghao Zhang
      21. HBASE-17125.master.016.patch
        34 kB
        Guanghao Zhang
      22. HBASE-17125.master.017.patch
        34 kB
        Guanghao Zhang
      23. HBASE-17125.master.018.patch
        35 kB
        Guanghao Zhang
      24. HBASE-17125.master.checkReturnedVersions.patch
        44 kB
        Guanghao Zhang
      25. HBASE-17125.master.no-specified-filter.patch
        39 kB
        Guanghao Zhang

        Issue Links

          Activity

          Hide
          zghaobac Guanghao Zhang added a comment -

          Upload a test example for this problem.

          Show
          zghaobac Guanghao Zhang added a comment - Upload a test example for this problem.
          Hide
          Apache9 Duo Zhang added a comment -

          I prefer the simple 'check versions then check filter' fix, but I'm not sure if some daughter projects need the old behavior.

          Andrew Purtell Does Phoenix need the old behavior where maxVersions is a limit for the whole result, not the result before filtering?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - I prefer the simple 'check versions then check filter' fix, but I'm not sure if some daughter projects need the old behavior. Andrew Purtell Does Phoenix need the old behavior where maxVersions is a limit for the whole result, not the result before filtering? Thanks.
          Hide
          apurtell Andrew Purtell added a comment -
          Show
          apurtell Andrew Purtell added a comment - James Taylor
          Hide
          Apache9 Duo Zhang added a comment -
          Show
          Apache9 Duo Zhang added a comment - Ping James Taylor .
          Hide
          apurtell Andrew Purtell added a comment -

          Let's wait a week due to the thanksgiving holiday in the US, then proceed as you like if still no response. We can run the Phoenix unit test suite with a patched HBase to check if they are obviously affected.

          Show
          apurtell Andrew Purtell added a comment - Let's wait a week due to the thanksgiving holiday in the US, then proceed as you like if still no response. We can run the Phoenix unit test suite with a patched HBase to check if they are obviously affected.
          Hide
          Apache9 Duo Zhang added a comment -

          Fine. Thanks.

          Show
          Apache9 Duo Zhang added a comment - Fine. Thanks.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          I think this was discussed previouly also. Lars Hofhansl had some similar thoughts. But I think not only down stream projects but many user apps may be running with the current logic. If we change it then it could have a bigger impact.
          May be we should introduce the new one as a toggle or by allowing user to configure this new ScanTracker? Am not sure on the best soln but just saying.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - I think this was discussed previouly also. Lars Hofhansl had some similar thoughts. But I think not only down stream projects but many user apps may be running with the current logic. If we change it then it could have a bigger impact. May be we should introduce the new one as a toggle or by allowing user to configure this new ScanTracker? Am not sure on the best soln but just saying.
          Hide
          Apache9 Duo Zhang added a comment -

          I think we can do it for 2.0 only.

          The old logic is not stable, I do not think we should let user depend on a broken semantic anymore, especially that we never guarantee this in the documentation.

          Show
          Apache9 Duo Zhang added a comment - I think we can do it for 2.0 only. The old logic is not stable, I do not think we should let user depend on a broken semantic anymore, especially that we never guarantee this in the documentation.
          Hide
          lhofhansl Lars Hofhansl added a comment - - edited

          Simply put: There is no right order in which to evaluate filters, versions, column tracking, and deletes. Regardless of which order you pick someone will complain about it.

          This has changed various times in the past. I think we should stick with how it is. If filters can see versions they can act accordingly (and filter extra versions if needed). If filters do not see all version, but you happen to need to look at all versions... Well then you'd be out of luck if we make this change.

          See also HBASE-13094. That one is about the order of filters w.r.t. delete markers.

          I guess, in hindsight for maximum flexibility filters should have been placed before everything else so that they can be used to intervene into the entire pipeline.

          Show
          lhofhansl Lars Hofhansl added a comment - - edited Simply put: There is no right order in which to evaluate filters, versions, column tracking, and deletes. Regardless of which order you pick someone will complain about it. This has changed various times in the past. I think we should stick with how it is. If filters can see versions they can act accordingly (and filter extra versions if needed). If filters do not see all version, but you happen to need to look at all versions... Well then you'd be out of luck if we make this change. See also HBASE-13094 . That one is about the order of filters w.r.t. delete markers. I guess, in hindsight for maximum flexibility filters should have been placed before everything else so that they can be used to intervene into the entire pipeline.
          Hide
          Apache9 Duo Zhang added a comment -

          HBASE-13094 aims to add a new type of filter right? This issue aims to solve the problem for the current fiter implementation.

          And I need to speak again, the old 'filter then check verions' is not stable. The same request can return different results only if there is a major compaction. This is a bug and we need to fix it.

          And the biggest problem here, is the semantic of setMaxVerions. In the javadoc of setFilter method, the max versions check should be evaluated before filtering but the current implementation is more likely a limitation of the final result. This leads to two approaches to fix the bug, one is try to break our javadoc and the new implementation will be more complicated, but can keep compatible with the old implementation as much as possible. The other is to change the implementation according to our javadoc, the fix is simple and straight-forward.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - HBASE-13094 aims to add a new type of filter right? This issue aims to solve the problem for the current fiter implementation. And I need to speak again, the old 'filter then check verions' is not stable. The same request can return different results only if there is a major compaction. This is a bug and we need to fix it. And the biggest problem here, is the semantic of setMaxVerions. In the javadoc of setFilter method, the max versions check should be evaluated before filtering but the current implementation is more likely a limitation of the final result. This leads to two approaches to fix the bug, one is try to break our javadoc and the new implementation will be more complicated, but can keep compatible with the old implementation as much as possible. The other is to change the implementation according to our javadoc, the fix is simple and straight-forward. Thanks.
          Hide
          huaxiang huaxiang sun added a comment -

          Hi Guanghao Zhang, "But this idea has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore." I think this is caused by the different meaning of scan.maxVersions and HCD's maxVersions? The implementation implies that they are same. Thanks.

          From doc:
          Scan setMaxVersions(int maxVersions)
          Get up to the specified number of versions of each column.

          The implementation:
          https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/UserScanQueryMatcher.java#L197

          Show
          huaxiang huaxiang sun added a comment - Hi Guanghao Zhang , "But this idea has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore." I think this is caused by the different meaning of scan.maxVersions and HCD's maxVersions? The implementation implies that they are same. Thanks. From doc: Scan setMaxVersions(int maxVersions) Get up to the specified number of versions of each column. The implementation: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/UserScanQueryMatcher.java#L197
          Hide
          stack stack added a comment -

          Marking as critical for 2.0. These 'unexpected behaviors' cause operator head-scratching and wasted hours of digging.

          Show
          stack stack added a comment - Marking as critical for 2.0. These 'unexpected behaviors' cause operator head-scratching and wasted hours of digging.
          Hide
          zghaobac Guanghao Zhang added a comment -

          I thought this should be fixed by the second idea.
          1. check by the max versions of this column, HCD's maxVersions
          2. check the kv by filter
          3. check the versions which user need, scan.maxVersions

          huaxiang sun Stack Andrew Purtell Lars Hofhansl ramkrishna.s.vasudevan Duo Zhang What do you think about this idea?

          Show
          zghaobac Guanghao Zhang added a comment - I thought this should be fixed by the second idea. 1. check by the max versions of this column, HCD's maxVersions 2. check the kv by filter 3. check the versions which user need, scan.maxVersions huaxiang sun Stack Andrew Purtell Lars Hofhansl ramkrishna.s.vasudevan Duo Zhang What do you think about this idea?
          Hide
          Apache9 Duo Zhang added a comment -

          I still think the simple fix is enough. 'Filter' is designed to move the computation from client to server, which implicitly means you can also implement the logic at client side. I think that's why our document says filter is tested at last, after ttl, column match, deletes and max versions.

          And I think we can introduce a new filter to provide the function that return specific versions after filter? Maybe called 'MaxVersionFilter'? You can set maxVersions to infinite and put MaxVersionFilter at the last position in FilterList to limit the versions returned to client.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - I still think the simple fix is enough. 'Filter' is designed to move the computation from client to server, which implicitly means you can also implement the logic at client side. I think that's why our document says filter is tested at last, after ttl, column match, deletes and max versions. And I think we can introduce a new filter to provide the function that return specific versions after filter? Maybe called 'MaxVersionFilter'? You can set maxVersions to infinite and put MaxVersionFilter at the last position in FilterList to limit the versions returned to client. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Upload a v1 patch. Discussed offline with Duo Zhang, we plan use the first simple idea to fix it. The ScanQueryMatcher will check the number of versions first, then check the cell by filter. This fix doesn't break the javadoc of Query.setFilter.

            /**
             * Apply the specified server-side filter when performing the Query.
             * Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests
             * for ttl, column match, deletes and max versions have been run.
             * @param filter filter to run on the server
             * @return this for invocation chaining
             */
            public Query setFilter(Filter filter) {
              this.filter = filter;
              return this;
            }
          
          Show
          zghaobac Guanghao Zhang added a comment - Upload a v1 patch. Discussed offline with Duo Zhang , we plan use the first simple idea to fix it. The ScanQueryMatcher will check the number of versions first, then check the cell by filter. This fix doesn't break the javadoc of Query.setFilter. /** * Apply the specified server-side filter when performing the Query. * Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests * for ttl, column match, deletes and max versions have been run. * @param filter filter to run on the server * @ return this for invocation chaining */ public Query setFilter(Filter filter) { this .filter = filter; return this ; }
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 6s master passed
          +1 compile 0m 36s master passed
          +1 checkstyle 0m 46s master passed
          +1 mvneclipse 0m 15s master passed
          +1 findbugs 1m 53s master passed
          +1 javadoc 0m 29s master passed
          +1 mvninstall 0m 41s the patch passed
          +1 compile 0m 40s the patch passed
          +1 javac 0m 40s the patch passed
          +1 checkstyle 0m 47s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 29m 21s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 11s the patch passed
          +1 javadoc 0m 28s the patch passed
          -1 unit 23m 23s hbase-server in the patch failed.
          +1 asflicense 0m 11s The patch does not generate ASF License warnings.
          65m 37s



          Reason Tests
          Failed junit tests hadoop.hbase.regionserver.TestMinVersions



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856950/HBASE-17125.master.001.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 3217dd7d6291 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 1160315
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6028/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6028/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 6s master passed +1 compile 0m 36s master passed +1 checkstyle 0m 46s master passed +1 mvneclipse 0m 15s master passed +1 findbugs 1m 53s master passed +1 javadoc 0m 29s master passed +1 mvninstall 0m 41s the patch passed +1 compile 0m 40s the patch passed +1 javac 0m 40s the patch passed +1 checkstyle 0m 47s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 29m 21s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 11s the patch passed +1 javadoc 0m 28s the patch passed -1 unit 23m 23s hbase-server in the patch failed. +1 asflicense 0m 11s The patch does not generate ASF License warnings. 65m 37s Reason Tests Failed junit tests hadoop.hbase.regionserver.TestMinVersions Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856950/HBASE-17125.master.001.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 3217dd7d6291 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 1160315 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6028/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6028/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Try to trigger Hadoop QA again.

          Show
          zghaobac Guanghao Zhang added a comment - Try to trigger Hadoop QA again.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 4m 24s master passed
          +1 compile 0m 46s master passed
          +1 checkstyle 0m 56s master passed
          +1 mvneclipse 0m 18s master passed
          +1 findbugs 2m 11s master passed
          +1 javadoc 0m 39s master passed
          +1 mvninstall 0m 55s the patch passed
          +1 compile 0m 47s the patch passed
          +1 javac 0m 47s the patch passed
          +1 checkstyle 0m 54s the patch passed
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 33m 13s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 6s the patch passed
          +1 javadoc 0m 37s the patch passed
          -1 unit 108m 25s hbase-server in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          157m 38s



          Reason Tests
          Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes
            hadoop.hbase.security.visibility.TestVisibilityLabelsWithCustomVisLabService
            hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService



          Subsystem Report/Notes
          Docker Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12857288/HBASE-17125.master.002.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux dc1065f7362a 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / e0365df
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6053/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6053/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 4m 24s master passed +1 compile 0m 46s master passed +1 checkstyle 0m 56s master passed +1 mvneclipse 0m 18s master passed +1 findbugs 2m 11s master passed +1 javadoc 0m 39s master passed +1 mvninstall 0m 55s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed +1 checkstyle 0m 54s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 33m 13s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 6s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 108m 25s hbase-server in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 157m 38s Reason Tests Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes   hadoop.hbase.security.visibility.TestVisibilityLabelsWithCustomVisLabService   hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService Subsystem Report/Notes Docker Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12857288/HBASE-17125.master.002.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux dc1065f7362a 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / e0365df Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6053/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6053/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          huaxiang huaxiang sun added a comment -

          Looks good to me. Maybe update the description of Scan and Get 's setMaxVersions() to make sure there is no confusion?

            /**
             * Get up to the specified number of versions of each column. 
              // Something like "set the maximum versions of each column the scan will check"?
             * @param maxVersions maximum versions for each column
             * @return this
             */
            public Scan setMaxVersions(int maxVersions) {
              this.maxVersions = maxVersions;
              return this;
            }
          
          
          Show
          huaxiang huaxiang sun added a comment - Looks good to me. Maybe update the description of Scan and Get 's setMaxVersions() to make sure there is no confusion? /** * Get up to the specified number of versions of each column. // Something like "set the maximum versions of each column the scan will check" ? * @param maxVersions maximum versions for each column * @ return this */ public Scan setMaxVersions( int maxVersions) { this .maxVersions = maxVersions; return this ; }
          Hide
          zghaobac Guanghao Zhang added a comment -

          huaxiang sun Thanks for point out this. The new implemention will break setMaxVersions's javadoc. I will update it in next patch.

          Show
          zghaobac Guanghao Zhang added a comment - huaxiang sun Thanks for point out this. The new implemention will break setMaxVersions's javadoc. I will update it in next patch.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Failed ut TestVisibilityLabelsWithDeletes is related.

          The reason is about the Delete the latest version of the specified column.

            /**
             * Delete the latest version of the specified column.
             * This is an expensive call in that on the server-side, it first does a
             * get to find the latest versions timestamp.  Then it adds a delete using
             * the fetched cells timestamp.
             * @param family family name
             * @param qualifier column qualifier
             * @return this for invocation chaining
             */
            public Delete addColumn(final byte [] family, final byte [] qualifier) {
              this.addColumn(family, qualifier, this.ts);
              return this;
            }
          

          In server-side, it will get the latest version. And it use setMaxVersions and DeleteVersionVisibilityExpressionFilter to get the latest version. But after this fix, setMaxVersions can not make sure to return the specified number of versions. So there is nothing to delete.
          Code in prePrepareTimeStampForDeleteVersion method of org.apache.hadoop.hbase.security.visibility.VisibilityController.

              get.setFilter(new DeleteVersionVisibilityExpressionFilter(visibilityTags,
                  VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT));
              List<Cell> result = ctx.getEnvironment().getRegion().get(get, false);
          
              if (result.size() < get.getMaxVersions()) {
                // Nothing to delete
                CellUtil.updateLatestStamp(cell, byteNow, 0);
                return;
              }
              if (result.size() > get.getMaxVersions()) {
                throw new RuntimeException("Unexpected size: " + result.size()
                    + ". Results more than the max versions obtained.");
              }
              Cell getCell = result.get(get.getMaxVersions() - 1);
              CellUtil.setTimestamp(cell, getCell.getTimestamp());
          
          Show
          zghaobac Guanghao Zhang added a comment - Failed ut TestVisibilityLabelsWithDeletes is related. The reason is about the Delete the latest version of the specified column. /** * Delete the latest version of the specified column. * This is an expensive call in that on the server-side, it first does a * get to find the latest versions timestamp. Then it adds a delete using * the fetched cells timestamp. * @param family family name * @param qualifier column qualifier * @ return this for invocation chaining */ public Delete addColumn( final byte [] family, final byte [] qualifier) { this .addColumn(family, qualifier, this .ts); return this ; } In server-side, it will get the latest version. And it use setMaxVersions and DeleteVersionVisibilityExpressionFilter to get the latest version. But after this fix, setMaxVersions can not make sure to return the specified number of versions. So there is nothing to delete. Code in prePrepareTimeStampForDeleteVersion method of org.apache.hadoop.hbase.security.visibility.VisibilityController. get.setFilter( new DeleteVersionVisibilityExpressionFilter(visibilityTags, VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT)); List<Cell> result = ctx.getEnvironment().getRegion().get(get, false ); if (result.size() < get.getMaxVersions()) { // Nothing to delete CellUtil.updateLatestStamp(cell, byteNow, 0); return ; } if (result.size() > get.getMaxVersions()) { throw new RuntimeException( "Unexpected size: " + result.size() + ". Results more than the max versions obtained." ); } Cell getCell = result.get(get.getMaxVersions() - 1); CellUtil.setTimestamp(cell, getCell.getTimestamp());
          Hide
          zghaobac Guanghao Zhang added a comment -

          Upload a 003 patch to fix the ut. The new patch will set max versions to Integer.MAX to get latest version and use count to get the latest timestamp. So add a param 'count' to VisibilityController#prePrepareTimeStampForDeleteVersion.

              // This get's max version is Integer.MAX
              get.setFilter(new DeleteVersionVisibilityExpressionFilter(visibilityTags,
                  VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT));
              List<Cell> result = ctx.getEnvironment().getRegion().get(get, false);
          
              if (result.size() < count) {
                // Nothing to delete
                CellUtil.updateLatestStamp(cell, byteNow, 0);
                return;
              }
              // Use 'count' to get the latest cell and update the timestamp of delete
              Cell getCell = result.get(count - 1);
              CellUtil.setTimestamp(cell, getCell.getTimestamp());
          

          Meanwhile, the patch update the javadoc of setMaxVersions in Scan and Get. And I add 2 ut about delete specified version of the specified column and delete the latest version of the specified column.

          Show
          zghaobac Guanghao Zhang added a comment - Upload a 003 patch to fix the ut. The new patch will set max versions to Integer.MAX to get latest version and use count to get the latest timestamp. So add a param 'count' to VisibilityController#prePrepareTimeStampForDeleteVersion. // This get's max version is Integer .MAX get.setFilter( new DeleteVersionVisibilityExpressionFilter(visibilityTags, VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT)); List<Cell> result = ctx.getEnvironment().getRegion().get(get, false ); if (result.size() < count) { // Nothing to delete CellUtil.updateLatestStamp(cell, byteNow, 0); return ; } // Use 'count' to get the latest cell and update the timestamp of delete Cell getCell = result.get(count - 1); CellUtil.setTimestamp(cell, getCell.getTimestamp()); Meanwhile, the patch update the javadoc of setMaxVersions in Scan and Get. And I add 2 ut about delete specified version of the specified column and delete the latest version of the specified column.
          Hide
          zghaobac Guanghao Zhang added a comment -

          ramkrishna.s.vasudevan Can you help to review the change about "Deletes with Cell Visibility"? Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - ramkrishna.s.vasudevan Can you help to review the change about "Deletes with Cell Visibility"? Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 3m 10s master passed
          +1 compile 0m 56s master passed
          +1 checkstyle 0m 47s master passed
          +1 mvneclipse 0m 25s master passed
          +1 findbugs 2m 41s master passed
          +1 javadoc 0m 46s master passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 1m 3s the patch passed
          +1 compile 0m 57s the patch passed
          +1 javac 0m 57s the patch passed
          +1 checkstyle 0m 47s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 29m 24s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 3m 3s the patch passed
          +1 javadoc 0m 46s the patch passed
          +1 unit 2m 19s hbase-client in the patch passed.
          -1 unit 100m 1s hbase-server in the patch failed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          149m 13s



          Reason Tests
          Failed junit tests hadoop.hbase.coprocessor.TestRegionObserverInterface



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858654/HBASE-17125.master.003.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux b72559fae53a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 44b2558
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6093/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6093/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 3m 10s master passed +1 compile 0m 56s master passed +1 checkstyle 0m 47s master passed +1 mvneclipse 0m 25s master passed +1 findbugs 2m 41s master passed +1 javadoc 0m 46s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 3s the patch passed +1 compile 0m 57s the patch passed +1 javac 0m 57s the patch passed +1 checkstyle 0m 47s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 29m 24s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 3m 3s the patch passed +1 javadoc 0m 46s the patch passed +1 unit 2m 19s hbase-client in the patch passed. -1 unit 100m 1s hbase-server in the patch failed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 149m 13s Reason Tests Failed junit tests hadoop.hbase.coprocessor.TestRegionObserverInterface Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858654/HBASE-17125.master.003.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux b72559fae53a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 44b2558 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6093/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6093/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Update javadoc of setMaxVersions in 005 patch.

          Show
          zghaobac Guanghao Zhang added a comment - Update javadoc of setMaxVersions in 005 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 28s Maven dependency ordering for branch
          +1 mvninstall 3m 14s master passed
          +1 compile 0m 53s master passed
          +1 checkstyle 0m 44s master passed
          +1 mvneclipse 0m 23s master passed
          +1 findbugs 2m 32s master passed
          +1 javadoc 0m 43s master passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 44s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 28m 3s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 3m 23s the patch passed
          +1 javadoc 0m 51s the patch passed
          +1 unit 2m 28s hbase-client in the patch passed.
          +1 unit 102m 31s hbase-server in the patch passed.
          +1 asflicense 0m 32s The patch does not generate ASF License warnings.
          150m 44s



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858821/HBASE-17125.master.004.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 1bc81a5f7bd0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / aace02a
          Default Java 1.8.0_121
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6101/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6101/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 28s Maven dependency ordering for branch +1 mvninstall 3m 14s master passed +1 compile 0m 53s master passed +1 checkstyle 0m 44s master passed +1 mvneclipse 0m 23s master passed +1 findbugs 2m 32s master passed +1 javadoc 0m 43s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 59s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 44s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 28m 3s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 3m 23s the patch passed +1 javadoc 0m 51s the patch passed +1 unit 2m 28s hbase-client in the patch passed. +1 unit 102m 31s hbase-server in the patch passed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 150m 44s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858821/HBASE-17125.master.004.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 1bc81a5f7bd0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / aace02a Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6101/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6101/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 22s Maven dependency ordering for branch
          +1 mvninstall 3m 1s master passed
          +1 compile 0m 51s master passed
          +1 checkstyle 0m 45s master passed
          +1 mvneclipse 0m 23s master passed
          +1 findbugs 2m 29s master passed
          +1 javadoc 0m 42s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 51s the patch passed
          +1 javac 0m 51s the patch passed
          +1 checkstyle 0m 43s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 26m 51s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 45s the patch passed
          +1 javadoc 0m 42s the patch passed
          +1 unit 2m 17s hbase-client in the patch passed.
          +1 unit 94m 43s hbase-server in the patch passed.
          +1 asflicense 0m 33s The patch does not generate ASF License warnings.
          140m 22s



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858829/HBASE-17125.master.005.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux c7876db4dcea 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / aace02a
          Default Java 1.8.0_121
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6103/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6103/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 22s Maven dependency ordering for branch +1 mvninstall 3m 1s master passed +1 compile 0m 51s master passed +1 checkstyle 0m 45s master passed +1 mvneclipse 0m 23s master passed +1 findbugs 2m 29s master passed +1 javadoc 0m 42s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 0m 59s the patch passed +1 compile 0m 51s the patch passed +1 javac 0m 51s the patch passed +1 checkstyle 0m 43s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 26m 51s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 45s the patch passed +1 javadoc 0m 42s the patch passed +1 unit 2m 17s hbase-client in the patch passed. +1 unit 94m 43s hbase-server in the patch passed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 140m 22s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858829/HBASE-17125.master.005.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux c7876db4dcea 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / aace02a Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6103/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6103/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Will take a look at the change in visibility specifcally. Thanks.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Will take a look at the change in visibility specifcally. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Ping Duo Zhang...

          Show
          zghaobac Guanghao Zhang added a comment - Ping Duo Zhang ...
          Hide
          zghaobac Guanghao Zhang added a comment -

          Open a issue HBASE-17797 to introduce a filter to implement read the special number of versions of column.

          Show
          zghaobac Guanghao Zhang added a comment - Open a issue HBASE-17797 to introduce a filter to implement read the special number of versions of column.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 3m 0s master passed
          +1 compile 0m 51s master passed
          +1 checkstyle 0m 43s master passed
          +1 mvneclipse 0m 23s master passed
          +1 findbugs 2m 30s master passed
          +1 javadoc 0m 41s master passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 0m 58s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 44s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 26m 59s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 45s the patch passed
          +1 javadoc 0m 42s the patch passed
          +1 unit 2m 14s hbase-client in the patch passed.
          +1 unit 99m 33s hbase-server in the patch passed.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          144m 59s



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859270/HBASE-17125.master.006.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 7392b7ea497a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 0ed50f4
          Default Java 1.8.0_121
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6130/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6130/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 3m 0s master passed +1 compile 0m 51s master passed +1 checkstyle 0m 43s master passed +1 mvneclipse 0m 23s master passed +1 findbugs 2m 30s master passed +1 javadoc 0m 41s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 58s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 44s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 26m 59s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 45s the patch passed +1 javadoc 0m 42s the patch passed +1 unit 2m 14s hbase-client in the patch passed. +1 unit 99m 33s hbase-server in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 144m 59s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859270/HBASE-17125.master.006.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 7392b7ea497a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 0ed50f4 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6130/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6130/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Stack Lars Hofhansl ramkrishna.s.vasudevan Can you help review? Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - Stack Lars Hofhansl ramkrishna.s.vasudevan Can you help review? Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -
          Show
          zghaobac Guanghao Zhang added a comment - ping ramkrishna.s.vasudevan
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          W.r.t to the change in the visibility controller it seems fine because instead of getting the maxVersions that was set in the Get/scan you are passing it now as the count param via the CPs..

          Set the maximum versions of each column will be check by filter. When use filter, the get

          257 * maybe return less than maximum versions for each column.

          Better to rephrase this like
          'Sets the maximum versions of each column that will be checked by the filters. When filters are in use, the get may return less than maximum versions for each column'.

          The rest of the patch need to see.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - W.r.t to the change in the visibility controller it seems fine because instead of getting the maxVersions that was set in the Get/scan you are passing it now as the count param via the CPs.. Set the maximum versions of each column will be check by filter. When use filter, the get 257 * maybe return less than maximum versions for each column. Better to rephrase this like 'Sets the maximum versions of each column that will be checked by the filters. When filters are in use, the get may return less than maximum versions for each column'. The rest of the patch need to see.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Joining here late.
          So this is a behave change right? To check for the versions first and then apply filter on top? There might be usages depending on the old behave?
          Just a use case. Am using visibility labels and having many values for the same key (Many cells with diff labels). A user having specific visibility label auth doing a get and it is a simple Get() request means want to see only one latest version for which I have label permission. Now after this patch, the versions is applied which will just give one version and that might not have a label corresponding to this user. So he will see no results! Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters. What to do here?

          Show
          anoop.hbase Anoop Sam John added a comment - Joining here late. So this is a behave change right? To check for the versions first and then apply filter on top? There might be usages depending on the old behave? Just a use case. Am using visibility labels and having many values for the same key (Many cells with diff labels). A user having specific visibility label auth doing a get and it is a simple Get() request means want to see only one latest version for which I have label permission. Now after this patch, the versions is applied which will just give one version and that might not have a label corresponding to this user. So he will see no results! Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters. What to do here?
          Hide
          zghaobac Guanghao Zhang added a comment -

          So this is a behave change right? To check for the versions first and then apply filter on top?

          Yeah, this patch will change setMaxVersions()'s behavior.

          Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters.

          I thought this can be fixed by HBASE-17797. When use auth and ACL, we can add a filter which return special number of versions for Get/Scan.

          Show
          zghaobac Guanghao Zhang added a comment - So this is a behave change right? To check for the versions first and then apply filter on top? Yeah, this patch will change setMaxVersions()'s behavior. Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters. I thought this can be fixed by HBASE-17797 . When use auth and ACL, we can add a filter which return special number of versions for Get/Scan.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Some crazy thinking.
          May be we should check the versions thing in 2 steps. We have max versions to keep per column in table definition. And every read request will have #versions that it would like to see.
          We should do the version number filter for the table def level versions# at first.
          Then Filter ops (As done today)
          Finally a check on the #versions as specified by the read op and return only specific #cells.
          Just wild thought. Dont know the feasibility and will add much more overhead or not.

          Show
          anoop.hbase Anoop Sam John added a comment - Some crazy thinking. May be we should check the versions thing in 2 steps. We have max versions to keep per column in table definition. And every read request will have #versions that it would like to see. We should do the version number filter for the table def level versions# at first. Then Filter ops (As done today) Finally a check on the #versions as specified by the read op and return only specific #cells. Just wild thought. Dont know the feasibility and will add much more overhead or not.
          Hide
          Apache9 Duo Zhang added a comment -

          Anoop Sam John We have considered this solution before. See this comment

          https://issues.apache.org/jira/browse/HBASE-17125?focusedCommentId=15889451&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15889451

          But I do not think it is necessary. Our documentation never says we will do things like this. It will make the implementation much more complicated. And if user still want this feature, it can be addressed by HBASE-17797.

          Here seems Guanghao Zhang want to address the problem of visivility labels and ACLs by always read all the versions. It will have bad impact on performance. So maybe we could close HBASE-17797 and just implement the max version filter here and give an example for how to use the filter?
          WDYT Guanghao Zhang?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Anoop Sam John We have considered this solution before. See this comment https://issues.apache.org/jira/browse/HBASE-17125?focusedCommentId=15889451&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15889451 But I do not think it is necessary. Our documentation never says we will do things like this. It will make the implementation much more complicated. And if user still want this feature, it can be addressed by HBASE-17797 . Here seems Guanghao Zhang want to address the problem of visivility labels and ACLs by always read all the versions. It will have bad impact on performance. So maybe we could close HBASE-17797 and just implement the max version filter here and give an example for how to use the filter? WDYT Guanghao Zhang ? Thanks.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Sorry did not read all the discuss in this jira. I came in late. As said just I was saying. Would like to see the patch for solving the ACL/labels issue.

          Show
          anoop.hbase Anoop Sam John added a comment - Sorry did not read all the discuss in this jira. I came in late. As said just I was saying. Would like to see the patch for solving the ACL/labels issue.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Add 007 patch which implement a new SpecifiedNumVersionsColumnFilter. And the ut testSpecifiedNumVersionsColumnFilter shows how to use this filter.

          Show
          zghaobac Guanghao Zhang added a comment - Add 007 patch which implement a new SpecifiedNumVersionsColumnFilter. And the ut testSpecifiedNumVersionsColumnFilter shows how to use this filter.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 31s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 43s Maven dependency ordering for branch
          +1 mvninstall 7m 39s master passed
          +1 compile 3m 32s master passed
          +1 checkstyle 20m 31s master passed
          +1 mvneclipse 0m 50s master passed
          -1 findbugs 4m 36s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 1m 32s master passed
          0 mvndep 0m 19s Maven dependency ordering for patch
          +1 mvninstall 3m 1s the patch passed
          +1 compile 2m 47s the patch passed
          +1 cc 2m 47s the patch passed
          +1 javac 2m 47s the patch passed
          +1 checkstyle 20m 35s the patch passed
          +1 mvneclipse 1m 0s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 61m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 2m 29s the patch passed
          +1 findbugs 13m 8s the patch passed
          +1 javadoc 1m 32s the patch passed
          +1 unit 0m 59s hbase-protocol-shaded in the patch passed.
          -1 unit 1m 47s hbase-client in the patch failed.
          -1 unit 168m 0s hbase-server in the patch failed.
          +1 asflicense 1m 13s The patch does not generate ASF License warnings.
          325m 22s



          Reason Tests
          Failed junit tests hadoop.hbase.TestInterfaceAudienceAnnotations
            hadoop.hbase.snapshot.TestExportSnapshot
            hadoop.hbase.client.TestAsyncBalancerAdminApi
            hadoop.hbase.snapshot.TestSecureExportSnapshot
            hadoop.hbase.snapshot.TestMobExportSnapshot
            hadoop.hbase.snapshot.TestMobSecureExportSnapshot
          Timed out junit tests org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
            org.apache.hadoop.hbase.master.TestMetaShutdownHandler



          Subsystem Report/Notes
          Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863201/HBASE-17125.master.007.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 6a6d517f01fc 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / da5fb27
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6410/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6410/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 43s Maven dependency ordering for branch +1 mvninstall 7m 39s master passed +1 compile 3m 32s master passed +1 checkstyle 20m 31s master passed +1 mvneclipse 0m 50s master passed -1 findbugs 4m 36s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 1m 32s master passed 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 3m 1s the patch passed +1 compile 2m 47s the patch passed +1 cc 2m 47s the patch passed +1 javac 2m 47s the patch passed +1 checkstyle 20m 35s the patch passed +1 mvneclipse 1m 0s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 61m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 2m 29s the patch passed +1 findbugs 13m 8s the patch passed +1 javadoc 1m 32s the patch passed +1 unit 0m 59s hbase-protocol-shaded in the patch passed. -1 unit 1m 47s hbase-client in the patch failed. -1 unit 168m 0s hbase-server in the patch failed. +1 asflicense 1m 13s The patch does not generate ASF License warnings. 325m 22s Reason Tests Failed junit tests hadoop.hbase.TestInterfaceAudienceAnnotations   hadoop.hbase.snapshot.TestExportSnapshot   hadoop.hbase.client.TestAsyncBalancerAdminApi   hadoop.hbase.snapshot.TestSecureExportSnapshot   hadoop.hbase.snapshot.TestMobExportSnapshot   hadoop.hbase.snapshot.TestMobSecureExportSnapshot Timed out junit tests org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence   org.apache.hadoop.hbase.master.TestMetaShutdownHandler Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863201/HBASE-17125.master.007.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 6a6d517f01fc 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / da5fb27 Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6410/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6410/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Add InterfaceAudience for SpecifiedNumVersionsColumnFilter. The failed ut passed locally. TestExportSnapshot* failed in other pre-commit jobs, too. So open a issue https://issues.apache.org/jira/browse/HBASE-17913 for it.

          Show
          zghaobac Guanghao Zhang added a comment - Add InterfaceAudience for SpecifiedNumVersionsColumnFilter. The failed ut passed locally. TestExportSnapshot* failed in other pre-commit jobs, too. So open a issue https://issues.apache.org/jira/browse/HBASE-17913 for it.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 30s Maven dependency ordering for branch
          +1 mvninstall 3m 17s master passed
          +1 compile 1m 31s master passed
          +1 checkstyle 10m 58s master passed
          +1 mvneclipse 0m 35s master passed
          -1 findbugs 2m 9s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 1m 0s master passed
          0 mvndep 0m 13s Maven dependency ordering for patch
          +1 mvninstall 1m 45s the patch passed
          +1 compile 1m 30s the patch passed
          +1 cc 1m 30s the patch passed
          +1 javac 1m 30s the patch passed
          +1 checkstyle 11m 16s the patch passed
          +1 mvneclipse 0m 32s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 29m 35s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 1m 5s the patch passed
          +1 findbugs 5m 50s the patch passed
          +1 javadoc 0m 54s the patch passed
          +1 unit 0m 29s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 18s hbase-client in the patch passed.
          -1 unit 109m 14s hbase-server in the patch failed.
          +1 asflicense 0m 48s The patch does not generate ASF License warnings.
          189m 49s



          Reason Tests
          Failed junit tests hadoop.hbase.snapshot.TestMobSecureExportSnapshot
            hadoop.hbase.snapshot.TestMobExportSnapshot
            hadoop.hbase.snapshot.TestSecureExportSnapshot



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863413/HBASE-17125.master.008.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 0cbf4c853473 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / da5fb27
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6420/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6420/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 30s Maven dependency ordering for branch +1 mvninstall 3m 17s master passed +1 compile 1m 31s master passed +1 checkstyle 10m 58s master passed +1 mvneclipse 0m 35s master passed -1 findbugs 2m 9s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 1m 0s master passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 45s the patch passed +1 compile 1m 30s the patch passed +1 cc 1m 30s the patch passed +1 javac 1m 30s the patch passed +1 checkstyle 11m 16s the patch passed +1 mvneclipse 0m 32s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 29m 35s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 1m 5s the patch passed +1 findbugs 5m 50s the patch passed +1 javadoc 0m 54s the patch passed +1 unit 0m 29s hbase-protocol-shaded in the patch passed. +1 unit 2m 18s hbase-client in the patch passed. -1 unit 109m 14s hbase-server in the patch failed. +1 asflicense 0m 48s The patch does not generate ASF License warnings. 189m 49s Reason Tests Failed junit tests hadoop.hbase.snapshot.TestMobSecureExportSnapshot   hadoop.hbase.snapshot.TestMobExportSnapshot   hadoop.hbase.snapshot.TestSecureExportSnapshot Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863413/HBASE-17125.master.008.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 0cbf4c853473 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / da5fb27 Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6420/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6420/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Anoop Sam John Sorry to upload patch late. Please help to review the 008 patch. Review board: https://reviews.apache.org/r/57449/ . Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - Anoop Sam John Sorry to upload patch late. Please help to review the 008 patch. Review board: https://reviews.apache.org/r/57449/ . Thanks.
          Hide
          Apache9 Duo Zhang added a comment -

          Ping Anoop Sam John. I'm +1 on the latest patch.

          Show
          Apache9 Duo Zhang added a comment - Ping Anoop Sam John . I'm +1 on the latest patch.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment - - edited

          has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore.

          How is the above addressed in the current patch ?

          Currently the max versions is obtained this way (see UserScanQueryMatcher):

              int maxVersions = scan.isRaw() ? scan.getMaxVersions()
                  : Math.min(scan.getMaxVersions(), scanInfo.getMaxVersions());
          

          The column tracker loses some information when column's max versions is greater than that specified in the Scan.
          Can we pass this information (let's call it the slack) to ColumnTracker ctor ?

          When filterResponse is SKIP, we can utilize the extra information to address the scenario described above by calling a new method of ColumnTracker (let's call it retract which decrements currentCount field if slack permits).

          Show
          yuzhihong@gmail.com Ted Yu added a comment - - edited has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore. How is the above addressed in the current patch ? Currently the max versions is obtained this way (see UserScanQueryMatcher): int maxVersions = scan.isRaw() ? scan.getMaxVersions() : Math .min(scan.getMaxVersions(), scanInfo.getMaxVersions()); The column tracker loses some information when column's max versions is greater than that specified in the Scan. Can we pass this information (let's call it the slack) to ColumnTracker ctor ? When filterResponse is SKIP, we can utilize the extra information to address the scenario described above by calling a new method of ColumnTracker (let's call it retract which decrements currentCount field if slack permits).
          Hide
          zghaobac Guanghao Zhang added a comment -

          How is the above addressed in the current patch ?

          Now scan.setMaxVerrsions means how many versions will be check. So this can be addressed by scan.setMaxVersions(5).

          Show
          zghaobac Guanghao Zhang added a comment - How is the above addressed in the current patch ? Now scan.setMaxVerrsions means how many versions will be check. So this can be addressed by scan.setMaxVersions(5).
          Hide
          zghaobac Guanghao Zhang added a comment -

          Can we pass this information (let's call it the slack) to ColumnTracker ctor ?

          The javadoc of scan.setMaxVersions has been changed. If user set max versions is less than the column's max versions, it means user didn't want to check all versions. So I thought we don't need this information?

          Show
          zghaobac Guanghao Zhang added a comment - Can we pass this information (let's call it the slack) to ColumnTracker ctor ? The javadoc of scan.setMaxVersions has been changed. If user set max versions is less than the column's max versions, it means user didn't want to check all versions. So I thought we don't need this information?
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Let's look at the scenario again:

          if a column's max version is 5 and the user query only need 3 versions

          If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned.
          Instead of letting user deal with the slack, it would be better to handle this server side.

          My proposal only involves a few lines of change to your latest patch - though there may be some unit test failure(s).

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Let's look at the scenario again: if a column's max version is 5 and the user query only need 3 versions If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned. Instead of letting user deal with the slack, it would be better to handle this server side. My proposal only involves a few lines of change to your latest patch - though there may be some unit test failure(s).
          Hide
          zghaobac Guanghao Zhang added a comment -

          If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned.

          This can be addressed by scan.setFilter(new SpecifiedNumVersionsColumnFilter(3)). setMaxVersions means how many version will be check. And SpecifiedNumVersionsColumnFilter means how many versions will be returned.

          Show
          zghaobac Guanghao Zhang added a comment - If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned. This can be addressed by scan.setFilter(new SpecifiedNumVersionsColumnFilter(3)). setMaxVersions means how many version will be check. And SpecifiedNumVersionsColumnFilter means how many versions will be returned.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          I dont think it is correct to ask users to do setMaxVersions(5). It will so complicated for a user to set this as 5 and then the filter with 3. This is like we pass our impl headache to user. IMHO.

          Show
          anoop.hbase Anoop Sam John added a comment - I dont think it is correct to ask users to do setMaxVersions(5). It will so complicated for a user to set this as 5 and then the filter with 3. This is like we pass our impl headache to user. IMHO.
          Hide
          Apache9 Duo Zhang added a comment -

          Why users need to setMaxVerions to 5 if they only want 3 versions? The fact is, if you do not use filter, then just use setMaxVersions to control the number of the veraions returned. If you use filter, and then please use SpecifiedNumVersionsColumnFilter if you want to control the number of versions returned as the max versions will be tested before filter. The setMaxVersions is used to control the number of versions passed to filter. I think this is clear enough?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Why users need to setMaxVerions to 5 if they only want 3 versions? The fact is, if you do not use filter, then just use setMaxVersions to control the number of the veraions returned. If you use filter, and then please use SpecifiedNumVersionsColumnFilter if you want to control the number of versions returned as the max versions will be tested before filter. The setMaxVersions is used to control the number of versions passed to filter. I think this is clear enough? Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          It will so complicated for a user to set this as 5 and then the filter with 3.

          The default version is 1. So the scan will only check the latest version. The user need set a bigger value if he want read more than one version. This scenario: the column's versions is 5. So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions.

          Show
          zghaobac Guanghao Zhang added a comment - It will so complicated for a user to set this as 5 and then the filter with 3. The default version is 1. So the scan will only check the latest version. The user need set a bigger value if he want read more than one version. This scenario: the column's versions is 5. So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions.
          Hide
          Apache9 Duo Zhang added a comment -

          So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions.

          Yes, just call scan.setMaxVerions(), do not need to give a specific value.

          Show
          Apache9 Duo Zhang added a comment - So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions. Yes, just call scan.setMaxVerions(), do not need to give a specific value.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          In user@ mailing list there was a query from a user regarding similar issue while using value filter. The recent value of a cell is not matching the value as per filter still he gets a older version of the cell.. He needs only one version (latest). So there also the user has to set this filter on get/scan and setMaxversions()?

          Show
          anoop.hbase Anoop Sam John added a comment - In user@ mailing list there was a query from a user regarding similar issue while using value filter. The recent value of a cell is not matching the value as per filter still he gets a older version of the cell.. He needs only one version (latest). So there also the user has to set this filter on get/scan and setMaxversions()?
          Hide
          Apache9 Duo Zhang added a comment -

          So there also the user has to set this filter on get/scan and setMaxversions()?

          No. After the patch here the problem is gone. Just keep the code, everything will be OK.

          And the problem of this fix is that we can not use setMaxVerions to control the number of returned versions if you user filter(maybe). So we introduce a SpecifiedNumVersionsColumnFilter to solve the problem.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - So there also the user has to set this filter on get/scan and setMaxversions()? No. After the patch here the problem is gone. Just keep the code, everything will be OK. And the problem of this fix is that we can not use setMaxVerions to control the number of returned versions if you user filter(maybe). So we introduce a SpecifiedNumVersionsColumnFilter to solve the problem. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          For the user mailing list case, the column's version is 1. So the user didn't need to setMaxVersions(). If the recent value is not matching, he will gets nothing.

          Show
          zghaobac Guanghao Zhang added a comment - For the user mailing list case, the column's version is 1. So the user didn't need to setMaxVersions(). If the recent value is not matching, he will gets nothing.
          Hide
          Apache9 Duo Zhang added a comment -

          Oh, seems the user calls setMaxVerions to 1. I believe the problem is that he/she found that the filter will return old values then he/she use setMaxVersions(1) and hope this could solve the problem.

          So it is clear that in this user's mind, setMaxVersions should be used to control the number of versions passed to the filter. This is exactly what we provide in the latest patch. With the patch in place, the user does not need to call setMaxVersions(1) anymore.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Oh, seems the user calls setMaxVerions to 1. I believe the problem is that he/she found that the filter will return old values then he/she use setMaxVersions(1) and hope this could solve the problem. So it is clear that in this user's mind, setMaxVersions should be used to control the number of versions passed to the filter. This is exactly what we provide in the latest patch. With the patch in place, the user does not need to call setMaxVersions(1) anymore. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          The right javadoc of setMaxVersions is:
          /**

          • If not use filter, get up to the specified number of versions of each column.
          • If use filter, it means the maximum versions of each column will be checked by filter. So the scan
          • maybe return less than maximum versions for each column. But you can add a SpecifiedNumVersionsColumnFilter
          • to get the specified number of versions of each column.
          • @param maxVersions maximum versions for each column
          • @return this
            */
          Show
          zghaobac Guanghao Zhang added a comment - The right javadoc of setMaxVersions is: /** If not use filter, get up to the specified number of versions of each column. If use filter, it means the maximum versions of each column will be checked by filter. So the scan maybe return less than maximum versions for each column. But you can add a SpecifiedNumVersionsColumnFilter to get the specified number of versions of each column. @param maxVersions maximum versions for each column @return this */
          Hide
          zghaobac Guanghao Zhang added a comment -

          Update the javadoc for setMaxVersions in 009 patch.

          Show
          zghaobac Guanghao Zhang added a comment - Update the javadoc for setMaxVersions in 009 patch.
          Hide
          Apache9 Duo Zhang added a comment -

          If not use filter, get up to the specified number of versions of each column.
          <p>
          If use filter, it means the maximum versions of each column will be checked by filter. So the scan may return less than the value you set here as the filter may filter out some cells. If you want to get a specific number of version for each column after filtering, please call setMaxVersions() and use SpecifiedNumVersionsColumnFilter. Notice that the SpecifiedNumVersionsColumnFilter should be placed at the last position in FilterList to make sure it will be checked at last.

          Better? And seems get also has this method so you need to change the wording from scan to get if possible?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - If not use filter, get up to the specified number of versions of each column. <p> If use filter, it means the maximum versions of each column will be checked by filter. So the scan may return less than the value you set here as the filter may filter out some cells. If you want to get a specific number of version for each column after filtering, please call setMaxVersions() and use SpecifiedNumVersionsColumnFilter. Notice that the SpecifiedNumVersionsColumnFilter should be placed at the last position in FilterList to make sure it will be checked at last. Better? And seems get also has this method so you need to change the wording from scan to get if possible? Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Update the javadoc in the latest 009 patch.

          Show
          zghaobac Guanghao Zhang added a comment - Update the javadoc in the latest 009 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 29s Maven dependency ordering for branch
          +1 mvninstall 3m 26s master passed
          +1 compile 1m 24s master passed
          +1 checkstyle 11m 2s master passed
          +1 mvneclipse 0m 32s master passed
          -1 findbugs 2m 5s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 0m 53s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 33s the patch passed
          +1 compile 1m 29s the patch passed
          +1 cc 1m 29s the patch passed
          +1 javac 1m 29s the patch passed
          +1 checkstyle 11m 2s the patch passed
          +1 mvneclipse 0m 31s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 28m 38s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 1m 2s the patch passed
          +1 findbugs 5m 33s the patch passed
          +1 javadoc 0m 50s the patch passed
          +1 unit 0m 26s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 16s hbase-client in the patch passed.
          -1 unit 112m 22s hbase-server in the patch failed.
          +1 asflicense 0m 48s The patch does not generate ASF License warnings.
          190m 27s



          Reason Tests
          Failed junit tests hadoop.hbase.regionserver.TestWalAndCompactingMemStoreFlush
            hadoop.hbase.snapshot.TestExportSnapshot



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864458/HBASE-17125.master.009.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 2f81ad5ac9e7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 33dadc1
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6526/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6526/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 29s Maven dependency ordering for branch +1 mvninstall 3m 26s master passed +1 compile 1m 24s master passed +1 checkstyle 11m 2s master passed +1 mvneclipse 0m 32s master passed -1 findbugs 2m 5s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 0m 53s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 33s the patch passed +1 compile 1m 29s the patch passed +1 cc 1m 29s the patch passed +1 javac 1m 29s the patch passed +1 checkstyle 11m 2s the patch passed +1 mvneclipse 0m 31s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 28m 38s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 1m 2s the patch passed +1 findbugs 5m 33s the patch passed +1 javadoc 0m 50s the patch passed +1 unit 0m 26s hbase-protocol-shaded in the patch passed. +1 unit 2m 16s hbase-client in the patch passed. -1 unit 112m 22s hbase-server in the patch failed. +1 asflicense 0m 48s The patch does not generate ASF License warnings. 190m 27s Reason Tests Failed junit tests hadoop.hbase.regionserver.TestWalAndCompactingMemStoreFlush   hadoop.hbase.snapshot.TestExportSnapshot Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864458/HBASE-17125.master.009.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 2f81ad5ac9e7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 33dadc1 Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6526/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6526/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          chia7712 Chia-Ping Tsai added a comment -

          TestWalAndCompactingMemStoreFlush is traced by HBASE-17943.

          Show
          chia7712 Chia-Ping Tsai added a comment - TestWalAndCompactingMemStoreFlush is traced by HBASE-17943 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 37s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 27s Maven dependency ordering for branch
          +1 mvninstall 3m 37s master passed
          +1 compile 1m 33s master passed
          +1 checkstyle 11m 42s master passed
          +1 mvneclipse 0m 34s master passed
          -1 findbugs 2m 15s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 0m 56s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 40s the patch passed
          +1 compile 1m 32s the patch passed
          +1 cc 1m 32s the patch passed
          +1 javac 1m 32s the patch passed
          +1 checkstyle 11m 43s the patch passed
          +1 mvneclipse 0m 34s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 33m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 1m 18s the patch passed
          +1 findbugs 6m 12s the patch passed
          +1 javadoc 0m 57s the patch passed
          +1 unit 0m 31s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 28s hbase-client in the patch passed.
          -1 unit 121m 53s hbase-server in the patch failed.
          +1 asflicense 0m 51s The patch does not generate ASF License warnings.
          208m 56s



          Reason Tests
          Failed junit tests hadoop.hbase.snapshot.TestMobExportSnapshot
            hadoop.hbase.snapshot.TestExportSnapshot
            hadoop.hbase.snapshot.TestMobSecureExportSnapshot



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864479/HBASE-17125.master.009.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 07fc238c0667 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / d39f40e
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6528/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6528/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 37s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 3m 37s master passed +1 compile 1m 33s master passed +1 checkstyle 11m 42s master passed +1 mvneclipse 0m 34s master passed -1 findbugs 2m 15s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 0m 56s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 40s the patch passed +1 compile 1m 32s the patch passed +1 cc 1m 32s the patch passed +1 javac 1m 32s the patch passed +1 checkstyle 11m 43s the patch passed +1 mvneclipse 0m 34s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 33m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 1m 18s the patch passed +1 findbugs 6m 12s the patch passed +1 javadoc 0m 57s the patch passed +1 unit 0m 31s hbase-protocol-shaded in the patch passed. +1 unit 2m 28s hbase-client in the patch passed. -1 unit 121m 53s hbase-server in the patch failed. +1 asflicense 0m 51s The patch does not generate ASF License warnings. 208m 56s Reason Tests Failed junit tests hadoop.hbase.snapshot.TestMobExportSnapshot   hadoop.hbase.snapshot.TestExportSnapshot   hadoop.hbase.snapshot.TestMobSecureExportSnapshot Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864479/HBASE-17125.master.009.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 07fc238c0667 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / d39f40e Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6528/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6528/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Apache9 Duo Zhang added a comment -

          'If not user filter,' -> 'If not use filter'.

          Any other concerns? Anoop Sam John.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - 'If not user filter,' -> 'If not use filter'. Any other concerns? Anoop Sam John . Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 27s Maven dependency ordering for branch
          +1 mvninstall 3m 42s master passed
          +1 compile 1m 32s master passed
          +1 checkstyle 11m 53s master passed
          +1 mvneclipse 0m 34s master passed
          -1 findbugs 2m 13s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 0m 58s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 40s the patch passed
          +1 compile 1m 32s the patch passed
          +1 cc 1m 32s the patch passed
          +1 javac 1m 32s the patch passed
          +1 checkstyle 11m 44s the patch passed
          +1 mvneclipse 0m 34s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 34m 17s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 1m 18s the patch passed
          +1 findbugs 6m 52s the patch passed
          +1 javadoc 0m 58s the patch passed
          +1 unit 0m 32s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 29s hbase-client in the patch passed.
          +1 unit 128m 59s hbase-server in the patch passed.
          +1 asflicense 0m 59s The patch does not generate ASF License warnings.
          217m 40s



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12866316/HBASE-17125.master.010.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 860de2b1a771 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 2de6b05
          Default Java 1.8.0_131
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6691/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6691/artifact/patchprocess/whitespace-eol.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6691/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6691/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 3m 42s master passed +1 compile 1m 32s master passed +1 checkstyle 11m 53s master passed +1 mvneclipse 0m 34s master passed -1 findbugs 2m 13s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 0m 58s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 40s the patch passed +1 compile 1m 32s the patch passed +1 cc 1m 32s the patch passed +1 javac 1m 32s the patch passed +1 checkstyle 11m 44s the patch passed +1 mvneclipse 0m 34s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 34m 17s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 1m 18s the patch passed +1 findbugs 6m 52s the patch passed +1 javadoc 0m 58s the patch passed +1 unit 0m 32s hbase-protocol-shaded in the patch passed. +1 unit 2m 29s hbase-client in the patch passed. +1 unit 128m 59s hbase-server in the patch passed. +1 asflicense 0m 59s The patch does not generate ASF License warnings. 217m 40s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12866316/HBASE-17125.master.010.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 860de2b1a771 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 2de6b05 Default Java 1.8.0_131 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6691/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6691/artifact/patchprocess/whitespace-eol.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6691/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6691/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Anoop Sam John Any more concerns?

          Show
          zghaobac Guanghao Zhang added a comment - Anoop Sam John Any more concerns?
          Hide
          zghaobac Guanghao Zhang added a comment -
          Show
          zghaobac Guanghao Zhang added a comment - Ping Anoop Sam John Lars Hofhansl stack ...
          Hide
          zghaobac Guanghao Zhang added a comment -

          Add a 011 patch which rebase the latest master branch. Anoop Sam John stack Any more concerns? If no objections, I will commit it tomorrow. Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - Add a 011 patch which rebase the latest master branch. Anoop Sam John stack Any more concerns? If no objections, I will commit it tomorrow. Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 14s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 27s Maven dependency ordering for branch
          +1 mvninstall 3m 48s master passed
          +1 compile 1m 56s master passed
          +1 checkstyle 16m 58s master passed
          +1 mvneclipse 0m 45s master passed
          -1 findbugs 2m 26s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 1m 19s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 43s the patch passed
          +1 compile 1m 35s the patch passed
          +1 cc 1m 35s the patch passed
          +1 javac 1m 35s the patch passed
          +1 checkstyle 12m 51s the patch passed
          +1 mvneclipse 0m 37s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 32m 42s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 hbaseprotoc 1m 12s the patch passed
          +1 findbugs 6m 23s the patch passed
          +1 javadoc 0m 7s hbase-protocol-shaded in the patch passed.
          +1 javadoc 0m 19s hbase-client generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2)
          +1 javadoc 0m 31s hbase-server in the patch passed.
          +1 unit 0m 32s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 41s hbase-client in the patch passed.
          -1 unit 125m 7s hbase-server in the patch failed.
          +1 asflicense 0m 54s The patch does not generate ASF License warnings.
          220m 29s



          Reason Tests
          Failed junit tests hadoop.hbase.security.token.TestZKSecretWatcher
            hadoop.hbase.master.procedure.TestMasterProcedureWalLease



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873450/HBASE-17125.master.011.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 2f4b61eb9eb8 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / ce1ce72
          Default Java 1.8.0_131
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7229/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7229/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 14s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 3m 48s master passed +1 compile 1m 56s master passed +1 checkstyle 16m 58s master passed +1 mvneclipse 0m 45s master passed -1 findbugs 2m 26s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 1m 19s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 43s the patch passed +1 compile 1m 35s the patch passed +1 cc 1m 35s the patch passed +1 javac 1m 35s the patch passed +1 checkstyle 12m 51s the patch passed +1 mvneclipse 0m 37s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 32m 42s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 hbaseprotoc 1m 12s the patch passed +1 findbugs 6m 23s the patch passed +1 javadoc 0m 7s hbase-protocol-shaded in the patch passed. +1 javadoc 0m 19s hbase-client generated 0 new + 1 unchanged - 1 fixed = 1 total (was 2) +1 javadoc 0m 31s hbase-server in the patch passed. +1 unit 0m 32s hbase-protocol-shaded in the patch passed. +1 unit 2m 41s hbase-client in the patch passed. -1 unit 125m 7s hbase-server in the patch failed. +1 asflicense 0m 54s The patch does not generate ASF License warnings. 220m 29s Reason Tests Failed junit tests hadoop.hbase.security.token.TestZKSecretWatcher   hadoop.hbase.master.procedure.TestMasterProcedureWalLease Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873450/HBASE-17125.master.011.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 2f4b61eb9eb8 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / ce1ce72 Default Java 1.8.0_131 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7229/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7229/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7229/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Show
          yuzhihong@gmail.com Ted Yu added a comment - For my previous comment: https://issues.apache.org/jira/browse/HBASE-17125?focusedCommentId=15976972&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15976972 which test in the patch exercises the above scenario ?
          Hide
          zghaobac Guanghao Zhang added a comment -

          which test in the patch exercises the above scenario ?

          The SpecifiedNumVersionsColumnFilter control how many versions column will be returned. Add a unit test TestFromClientSide#testSpecifiedNumVersionsColumnFilter for it. Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - which test in the patch exercises the above scenario ? The SpecifiedNumVersionsColumnFilter control how many versions column will be returned. Add a unit test TestFromClientSide#testSpecifiedNumVersionsColumnFilter for it. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          The findbugs and whitespace were introduced by generated protobuf. And the failed tests were not rleated. They passed locally. Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - The findbugs and whitespace were introduced by generated protobuf. And the failed tests were not rleated. They passed locally. Thanks.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Here is snippet showing the concept of slack:
          https://pastebin.com/diBC8B9M

          In UserScanQueryMatcher, around line 163, the call to retract() should be added:

                switch (filterResponse) {
                  case SKIP:
                    if (colChecker == MatchCode.INCLUDE) {
                      columns.retract();
                      return MatchCode.SKIP;
          

          The above is tentative - certain unit tests don't pass yet.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Here is snippet showing the concept of slack: https://pastebin.com/diBC8B9M In UserScanQueryMatcher, around line 163, the call to retract() should be added: switch (filterResponse) { case SKIP: if (colChecker == MatchCode.INCLUDE) { columns.retract(); return MatchCode.SKIP; The above is tentative - certain unit tests don't pass yet.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          IMHO we should address this issue on our own (Like what Ted is trying to suggest here). Am still not in favor of asking the user to configure some extra Filter to get an expected behave from the system. Can we pls think in that direction too. Ya ways were mentioned above also.

          Show
          anoop.hbase Anoop Sam John added a comment - IMHO we should address this issue on our own (Like what Ted is trying to suggest here). Am still not in favor of asking the user to configure some extra Filter to get an expected behave from the system. Can we pls think in that direction too. Ya ways were mentioned above also.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Here is snippet showing the concept of slack:

          IMHO we should address this issue on our own (Like what Ted is trying to suggest here).

          Ok. Let me try it.

          Show
          zghaobac Guanghao Zhang added a comment - Here is snippet showing the concept of slack: IMHO we should address this issue on our own (Like what Ted is trying to suggest here). Ok. Let me try it.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          When I tested my tentative patch, TestImportTSVWithVisibilityLabels hung.

          TestImportTSVWithVisibilityLabels#testBulkOutputWithTsvImporterTextMapperWithInvalidLabels seems to be the last subtest running before I killed the surefire process.

          FYI

          Show
          yuzhihong@gmail.com Ted Yu added a comment - When I tested my tentative patch, TestImportTSVWithVisibilityLabels hung. TestImportTSVWithVisibilityLabels#testBulkOutputWithTsvImporterTextMapperWithInvalidLabels seems to be the last subtest running before I killed the surefire process. FYI
          Hide
          Apache9 Duo Zhang added a comment -

          Am still not in favor of asking the user to configure some extra Filter to get an expected behave from the system

          I'd say again that the javadoc never guarantee the current behavior and no doubt it is a broken semantic. And see my comment above, I think use another filter to address the problem introduced by filter is the right direction. We should not put too many complexities to our core system.

          And see my comment above, a real user case which shows that the current approach can solve his/her problem

          Oh, seems the user calls setMaxVerions to 1. I believe the problem is that he/she found that the filter will return old values then he/she use setMaxVersions(1) and hope this could solve the problem.
          So it is clear that in this user's mind, setMaxVersions should be used to control the number of versions passed to the filter. This is exactly what we provide in the latest patch. With the patch in place, the user does not need to call setMaxVersions(1) anymore.
          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Am still not in favor of asking the user to configure some extra Filter to get an expected behave from the system I'd say again that the javadoc never guarantee the current behavior and no doubt it is a broken semantic. And see my comment above, I think use another filter to address the problem introduced by filter is the right direction. We should not put too many complexities to our core system. And see my comment above, a real user case which shows that the current approach can solve his/her problem Oh, seems the user calls setMaxVerions to 1. I believe the problem is that he/she found that the filter will return old values then he/she use setMaxVersions(1) and hope this could solve the problem. So it is clear that in this user's mind, setMaxVersions should be used to control the number of versions passed to the filter. This is exactly what we provide in the latest patch. With the patch in place, the user does not need to call setMaxVersions(1) anymore. Thanks.
          Hide
          Apache9 Duo Zhang added a comment -

          And we have already speak out at HBaseCon, there is no big concerns. I think most users just do not care about it as usually we only have one version, just like the user asked on mailing list.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - And we have already speak out at HBaseCon, there is no big concerns. I think most users just do not care about it as usually we only have one version, just like the user asked on mailing list. Thanks.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          The reasons Anoop and I don't favor SpecifiedNumVersionsColumnFilter are:

          • it is not intuitive.
          • user may use it incorrectly (conside nested FilterLists).

          Guanghao has agreed to try the new approach which doesn't add much more complexity on top of what is posted so far. Let's give him some time.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - The reasons Anoop and I don't favor SpecifiedNumVersionsColumnFilter are: it is not intuitive. user may use it incorrectly (conside nested FilterLists). Guanghao has agreed to try the new approach which doesn't add much more complexity on top of what is posted so far. Let's give him some time.
          Hide
          Apache9 Duo Zhang added a comment -

          We have already discussed many times on how to implement it, either in our company or here.

          which doesn't add much more complexity

          Then why don't you implement it by yourself? If you think it is easy, then please implement it.

          And your approach will lose the ability to control the versions passed to filter. And it can not be addressed by adding a new filter at the beginning, because you will always reset the version count if the filter list returns SKIP.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - We have already discussed many times on how to implement it, either in our company or here. which doesn't add much more complexity Then why don't you implement it by yourself? If you think it is easy, then please implement it. And your approach will lose the ability to control the versions passed to filter. And it can not be addressed by adding a new filter at the beginning, because you will always reset the version count if the filter list returns SKIP. Thanks.
          Hide
          Apache9 Duo Zhang added a comment -

          And I need to say again, we have never break the javadoc of setMaxVersions. If you do not use filtr, then the behavior is correct. If you use filter, then this is what the javadoc of setFilter says, filter is tested at last.

          /**

          • Apply the specified server-side filter when performing the Query.
          • Only
            Unknown macro: {@link Filter#filterKeyValue(org.apache.hadoop.hbase.Cell)}

            is called AFTER all tests

          • for ttl, column match, deletes and max versions have been run.
          • @param filter filter to run on the server
          • @return this for invocation chaining
            */

          Filter always introduces behaviors which are not intuitive because it is too flexible. For example, PageFilter may still return more rows than configured. You need to know the details of HBase if you want to use filter correctly. That's why I want to fix the problem in a simpler way rather than a complicated way. It does not make any big difference to users. For normal users they just do not care, and for advanced users they must know the implementation details.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - And I need to say again, we have never break the javadoc of setMaxVersions. If you do not use filtr, then the behavior is correct. If you use filter, then this is what the javadoc of setFilter says, filter is tested at last. /** Apply the specified server-side filter when performing the Query. Only Unknown macro: {@link Filter#filterKeyValue(org.apache.hadoop.hbase.Cell)} is called AFTER all tests for ttl, column match, deletes and max versions have been run. @param filter filter to run on the server @return this for invocation chaining */ Filter always introduces behaviors which are not intuitive because it is too flexible. For example, PageFilter may still return more rows than configured. You need to know the details of HBase if you want to use filter correctly. That's why I want to fix the problem in a simpler way rather than a complicated way. It does not make any big difference to users. For normal users they just do not care, and for advanced users they must know the implementation details. Thanks.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          What Ted 2 reasons why I fear abt the way of asking new Filter usage.. I did not check his suggestion fully wrt code and side effects if any.. But IMHO we should try it out.. With out much perf penalty if we can do, why not! If the code itself can handle this that is always better and users will be much more happy(Than they have to remember and use another filter).. Lets be open for all possible ways. My request he would be pls see possible ways with which we can solve it on our own.. All agree here that the present situation is a bug and really bad behave from system. We never give back deterministic results. (All depends on other factors like compaction done or not etc)

          Show
          anoop.hbase Anoop Sam John added a comment - What Ted 2 reasons why I fear abt the way of asking new Filter usage.. I did not check his suggestion fully wrt code and side effects if any.. But IMHO we should try it out.. With out much perf penalty if we can do, why not! If the code itself can handle this that is always better and users will be much more happy(Than they have to remember and use another filter).. Lets be open for all possible ways. My request he would be pls see possible ways with which we can solve it on our own.. All agree here that the present situation is a bug and really bad behave from system. We never give back deterministic results. (All depends on other factors like compaction done or not etc)
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Agree Duo.. We say in javadoc when we will call filter. But if we can solve this issue in our code itself, that is great. Ur concern of more complexity also valid only.. My request would be lets try that approach (I dont mind who do ) also and see.. How complex or not it is. Any perf impact or not.

          Show
          anoop.hbase Anoop Sam John added a comment - Agree Duo.. We say in javadoc when we will call filter. But if we can solve this issue in our code itself, that is great. Ur concern of more complexity also valid only.. My request would be lets try that approach (I dont mind who do ) also and see.. How complex or not it is. Any perf impact or not.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          I was following this discussion previously and lost track. I have seen this problem consistently arising. In all the different JIRAs related to this the fix inside the core was always having some side effects one way or the other.

          For the Visibility labels and ACL comment from Anoop, I think if we remove the Logic in VisiblityLabelFilter's filterKV to get the max number of versions and just add the new filter at the end in VisibiityController and set scan.setMaxVersion() all the visibilty tests should still pass. Can you try it out once Guanghao Zhang?

          I think having a javadoc and a new filter for the user in case where he really wants multiple versions to be returned back even while filtering means he can try using the new filter provided (if it is covering all the cases) as the core side changes are minimal and most importantly bug free.
          IMHO - it is fine with me to add a new filter and expose it to user for those who need this filter + version specific usage.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - I was following this discussion previously and lost track. I have seen this problem consistently arising. In all the different JIRAs related to this the fix inside the core was always having some side effects one way or the other. For the Visibility labels and ACL comment from Anoop, I think if we remove the Logic in VisiblityLabelFilter's filterKV to get the max number of versions and just add the new filter at the end in VisibiityController and set scan.setMaxVersion() all the visibilty tests should still pass. Can you try it out once Guanghao Zhang ? I think having a javadoc and a new filter for the user in case where he really wants multiple versions to be returned back even while filtering means he can try using the new filter provided (if it is covering all the cases) as the core side changes are minimal and most importantly bug free. IMHO - it is fine with me to add a new filter and expose it to user for those who need this filter + version specific usage.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Can you try it out once Guanghao Zhang?

          Ok. I will try to trigger the Hadoop QA again.

          Show
          zghaobac Guanghao Zhang added a comment - Can you try it out once Guanghao Zhang? Ok. I will try to trigger the Hadoop QA again.
          Hide
          zghaobac Guanghao Zhang added a comment -

          My request would be lets try that approach

          I attach a HBASE-17125.master.no-specified-filter.patch for it. The new approach still check versions first, then check by filter. And ColumnTracker use a returnedVersions to track how many versions will be returned. So checkVerisons will check max version by column family's versions. And the returned versions will controlled by user's scan's maxVersions. If the filter decide to skip a version, then reduce the returned count in ColumnTracker.

          Show
          zghaobac Guanghao Zhang added a comment - My request would be lets try that approach I attach a HBASE-17125 .master.no-specified-filter.patch for it. The new approach still check versions first, then check by filter. And ColumnTracker use a returnedVersions to track how many versions will be returned. So checkVerisons will check max version by column family's versions. And the returned versions will controlled by user's scan's maxVersions. If the filter decide to skip a version, then reduce the returned count in ColumnTracker.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment - - edited

          When I ran TestImportTSVWithVisibilityLabels with the no-specified-filter patch, it hung.
          On Linux, the test passes.

          TestVisibilityLabelsWithDeletes fails (as shown by QA).

          Please put HBASE-17125.master.no-specified-filter.patch on review board.

          Thanks

          Show
          yuzhihong@gmail.com Ted Yu added a comment - - edited When I ran TestImportTSVWithVisibilityLabels with the no-specified-filter patch, it hung. On Linux, the test passes. TestVisibilityLabelsWithDeletes fails (as shown by QA). Please put HBASE-17125 .master.no-specified-filter.patch on review board. Thanks
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 13m 59s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 4m 40s master passed
          +1 compile 0m 50s master passed
          +1 checkstyle 0m 56s master passed
          +1 mvneclipse 0m 18s master passed
          -1 findbugs 3m 15s hbase-server in master has 12 extant Findbugs warnings.
          +1 javadoc 0m 37s master passed
          +1 mvninstall 0m 56s the patch passed
          +1 compile 0m 50s the patch passed
          +1 javac 0m 50s the patch passed
          +1 checkstyle 0m 55s the patch passed
          +1 mvneclipse 0m 17s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 31m 31s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 2m 44s the patch passed
          +1 javadoc 0m 26s the patch passed
          -1 unit 119m 48s hbase-server in the patch failed.
          +1 asflicense 0m 25s The patch does not generate ASF License warnings.
          183m 5s



          Reason Tests
          Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes



          Subsystem Report/Notes
          Docker Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873837/HBASE-17125.master.no-specified-filter.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux ffcf4b97ea45 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 83be50c
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7275/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7275/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7275/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7275/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7275/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 13m 59s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 4m 40s master passed +1 compile 0m 50s master passed +1 checkstyle 0m 56s master passed +1 mvneclipse 0m 18s master passed -1 findbugs 3m 15s hbase-server in master has 12 extant Findbugs warnings. +1 javadoc 0m 37s master passed +1 mvninstall 0m 56s the patch passed +1 compile 0m 50s the patch passed +1 javac 0m 50s the patch passed +1 checkstyle 0m 55s the patch passed +1 mvneclipse 0m 17s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 31m 31s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 2m 44s the patch passed +1 javadoc 0m 26s the patch passed -1 unit 119m 48s hbase-server in the patch failed. +1 asflicense 0m 25s The patch does not generate ASF License warnings. 183m 5s Reason Tests Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes Subsystem Report/Notes Docker Client=1.11.2 Server=1.11.2 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873837/HBASE-17125.master.no-specified-filter.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux ffcf4b97ea45 3.13.0-116-generic #163-Ubuntu SMP Fri Mar 31 14:13:22 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 83be50c Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7275/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html unit https://builds.apache.org/job/PreCommit-HBASE-Build/7275/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7275/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7275/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7275/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 12m 29s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          +1 mvninstall 3m 40s master passed
          +1 compile 0m 39s master passed
          +1 checkstyle 0m 47s master passed
          +1 mvneclipse 0m 14s master passed
          -1 findbugs 2m 34s hbase-server in master has 12 extant Findbugs warnings.
          +1 javadoc 0m 28s master passed
          +1 mvninstall 0m 44s the patch passed
          +1 compile 0m 38s the patch passed
          +1 javac 0m 38s the patch passed
          +1 checkstyle 0m 48s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 31m 9s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 3m 18s the patch passed
          +1 javadoc 0m 31s the patch passed
          -1 unit 124m 14s hbase-server in the patch failed.
          +1 asflicense 0m 22s The patch does not generate ASF License warnings.
          183m 11s



          Reason Tests
          Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873837/HBASE-17125.master.no-specified-filter.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 4238af0c8af6 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 83be50c
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7276/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7276/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7276/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7276/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7276/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 12m 29s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. +1 mvninstall 3m 40s master passed +1 compile 0m 39s master passed +1 checkstyle 0m 47s master passed +1 mvneclipse 0m 14s master passed -1 findbugs 2m 34s hbase-server in master has 12 extant Findbugs warnings. +1 javadoc 0m 28s master passed +1 mvninstall 0m 44s the patch passed +1 compile 0m 38s the patch passed +1 javac 0m 38s the patch passed +1 checkstyle 0m 48s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 31m 9s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 3m 18s the patch passed +1 javadoc 0m 31s the patch passed -1 unit 124m 14s hbase-server in the patch failed. +1 asflicense 0m 22s The patch does not generate ASF License warnings. 183m 11s Reason Tests Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873837/HBASE-17125.master.no-specified-filter.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 4238af0c8af6 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 83be50c Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7276/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html unit https://builds.apache.org/job/PreCommit-HBASE-Build/7276/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7276/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7276/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7276/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          stevel@apache.org Steve Loughran added a comment -

          Then why don't you implement it by yourself? If you think it is easy, then please implement it.

          Generally, as a committer it's actually more productive to nurture other developers into working towards what you believe to be the right answer than do it yourself. As well as sharing some of your unrealistic set of deliverables with others, you can be the reviewer to gets the stuff in, instead of having a patch you have chase other people to review. Long term: the more people you can you can get to collaborate helps the project all round.

          No opinions on the patch, just making sure everyone works together on this. Thanks.

          Show
          stevel@apache.org Steve Loughran added a comment - Then why don't you implement it by yourself? If you think it is easy, then please implement it. Generally, as a committer it's actually more productive to nurture other developers into working towards what you believe to be the right answer than do it yourself. As well as sharing some of your unrealistic set of deliverables with others, you can be the reviewer to gets the stuff in, instead of having a patch you have chase other people to review. Long term: the more people you can you can get to collaborate helps the project all round. No opinions on the patch, just making sure everyone works together on this. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          If the filter decide to skip a version, then reduce the returned count in ColumnTracker.

          This method is too trick. And it is easy to have bug. So I upload a new patch (checkReturnedVersions.patch) which use the second idea in the description.
          It have three steps to match column.
          1. check the column family's max versions.
          2. check by filter
          3. check the returned versions. (This can be set by user).

          About the setFilter()'s javadoc. It says "called AFTER all tests
          for ttl, column match, deletes and max versions have been run." Talked with Phil Yang and Duo Zhang, we thought the max versions is easy to misunderstanding. Because the column family has a max versions config and user can set a max versions to scan. So in the new patch, I update the javadoc of setFilter() method. The new javadoc is "called AFTER all tests for ttl, column match, deletes and column family's max versions have been run". And add a new method setVersions() for scan, which means how many versions will be returned to user. And add a @deprecated mark for setMaxVersions() method. Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - If the filter decide to skip a version, then reduce the returned count in ColumnTracker. This method is too trick. And it is easy to have bug. So I upload a new patch (checkReturnedVersions.patch) which use the second idea in the description. It have three steps to match column. 1. check the column family's max versions. 2. check by filter 3. check the returned versions. (This can be set by user). About the setFilter()'s javadoc. It says "called AFTER all tests for ttl, column match, deletes and max versions have been run." Talked with Phil Yang and Duo Zhang , we thought the max versions is easy to misunderstanding. Because the column family has a max versions config and user can set a max versions to scan. So in the new patch, I update the javadoc of setFilter() method. The new javadoc is "called AFTER all tests for ttl, column match, deletes and column family's max versions have been run". And add a new method setVersions() for scan, which means how many versions will be returned to user. And add a @deprecated mark for setMaxVersions() method. Thanks.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Can you put the latest patch on review board ?

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Can you put the latest patch on review board ?
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 18m 48s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 11s Maven dependency ordering for branch
          +1 mvninstall 4m 6s master passed
          +1 compile 1m 2s master passed
          +1 checkstyle 0m 50s master passed
          +1 mvneclipse 0m 27s master passed
          -1 findbugs 0m 58s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 2m 59s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 0m 52s master passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 9s the patch passed
          +1 compile 1m 3s the patch passed
          +1 javac 1m 3s the patch passed
          +1 checkstyle 0m 49s the patch passed
          +1 mvneclipse 0m 26s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 32m 58s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 5m 6s the patch passed
          -1 javadoc 0m 19s hbase-client generated 5 new + 1 unchanged - 0 fixed = 6 total (was 1)
          +1 unit 2m 46s hbase-client in the patch passed.
          -1 unit 22m 42s hbase-server in the patch failed.
          +1 asflicense 0m 19s The patch does not generate ASF License warnings.
          99m 12s



          Reason Tests
          Failed junit tests hadoop.hbase.regionserver.TestKeepDeletes
            hadoop.hbase.regionserver.querymatcher.TestUserScanQueryMatcher



          Subsystem Report/Notes
          Docker Client=1.13.1 Server=1.13.1 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873971/HBASE-17125.master.checkReturnedVersions.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 0c417fc24366 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 3489a1b
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7285/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7285/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 18m 48s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 11s Maven dependency ordering for branch +1 mvninstall 4m 6s master passed +1 compile 1m 2s master passed +1 checkstyle 0m 50s master passed +1 mvneclipse 0m 27s master passed -1 findbugs 0m 58s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 2m 59s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 0m 52s master passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 9s the patch passed +1 compile 1m 3s the patch passed +1 javac 1m 3s the patch passed +1 checkstyle 0m 49s the patch passed +1 mvneclipse 0m 26s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 32m 58s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 5m 6s the patch passed -1 javadoc 0m 19s hbase-client generated 5 new + 1 unchanged - 0 fixed = 6 total (was 1) +1 unit 2m 46s hbase-client in the patch passed. -1 unit 22m 42s hbase-server in the patch failed. +1 asflicense 0m 19s The patch does not generate ASF License warnings. 99m 12s Reason Tests Failed junit tests hadoop.hbase.regionserver.TestKeepDeletes   hadoop.hbase.regionserver.querymatcher.TestUserScanQueryMatcher Subsystem Report/Notes Docker Client=1.13.1 Server=1.13.1 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12873971/HBASE-17125.master.checkReturnedVersions.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 0c417fc24366 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 3489a1b Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7285/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7285/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7285/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Apache9 Duo Zhang added a comment -

          I think we can append a SpecificNumberVersionsFilter at the last if filter and versions are both present when initialize a scan at RS side. This way we do not need to modify the logic of ColumnTracker, the code is already complicated enough so do not put new stuffs to it. I still think the problem introduced by filter should also be addressed by filter.

          And the name 'setVersions' does not make sense, change it to 'readAllVersions'? And how do you deal with raw scan?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - I think we can append a SpecificNumberVersionsFilter at the last if filter and versions are both present when initialize a scan at RS side. This way we do not need to modify the logic of ColumnTracker, the code is already complicated enough so do not put new stuffs to it. I still think the problem introduced by filter should also be addressed by filter. And the name 'setVersions' does not make sense, change it to 'readAllVersions'? And how do you deal with raw scan? Thanks.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          I think we can append a SpecificNumberVersionsFilter at the last if filter and versions are both present when initialize a scan at RS side.

          U mean HBase code itself add it than asking the user to do this right? Am fine with any approach (which seems out to be the best by all of us) which can help to solve this bug by HBase code itself. (Than asking user to do some extra work).. If this is really not possible, then only , as a last step ask the user to do some extra things.
          Will look at the patches and approaches. Thanks guys

          Show
          anoop.hbase Anoop Sam John added a comment - I think we can append a SpecificNumberVersionsFilter at the last if filter and versions are both present when initialize a scan at RS side. U mean HBase code itself add it than asking the user to do this right? Am fine with any approach (which seems out to be the best by all of us) which can help to solve this bug by HBase code itself. (Than asking user to do some extra work).. If this is really not possible, then only , as a last step ask the user to do some extra things. Will look at the patches and approaches. Thanks guys
          Hide
          Apache9 Duo Zhang added a comment -

          Yeah I mean do it by ourselves, not by users.

          Show
          Apache9 Duo Zhang added a comment - Yeah I mean do it by ourselves, not by users.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          That makes sense Duo.. Already we have a FilterWrapper been used to wrap user side filters. Pls check how this idea will turn out to be.. Thanks a lot Guanghao for ur perseverance. Appreciate it !

          Show
          anoop.hbase Anoop Sam John added a comment - That makes sense Duo.. Already we have a FilterWrapper been used to wrap user side filters. Pls check how this idea will turn out to be.. Thanks a lot Guanghao for ur perseverance. Appreciate it !
          Hide
          yuzhihong@gmail.com Ted Yu added a comment - - edited

          Based on patch v11 and the snippet I posted earlier, I have an aggregate patch which passes all Filter tests and the visibility test.
          The complexity of the aggregate patch is on par with patch v11 (sans the SpecifiedNumVersionsColumnFilter).

          Running thru whole test suite now.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - - edited Based on patch v11 and the snippet I posted earlier, I have an aggregate patch which passes all Filter tests and the visibility test. The complexity of the aggregate patch is on par with patch v11 (sans the SpecifiedNumVersionsColumnFilter). Running thru whole test suite now.
          Hide
          yangzhe1991 Phil Yang added a comment -

          I think setVersions may still confused us and users. Call it setMaxVersionsAfterFilters? And I think we should add some comments here to tell users how we deal with cf's VERSIONS, filters and Scan#setMaxVersions

          Show
          yangzhe1991 Phil Yang added a comment - I think setVersions may still confused us and users. Call it setMaxVersionsAfterFilters? And I think we should add some comments here to tell users how we deal with cf's VERSIONS, filters and Scan#setMaxVersions
          Hide
          Apache9 Duo Zhang added a comment -

          But for most users they just do not use filter so I do not think it is a good idea to add the word 'filter' to the method name.

          Show
          Apache9 Duo Zhang added a comment - But for most users they just do not use filter so I do not think it is a good idea to add the word 'filter' to the method name.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Take a summary about the key points of this fix.
          1. Easy to use for user, no need extra work.
          2. No more complicated work for ColumnTracker. Only fix this when use filter.
          3. Accurate javadoc. And mark scan's setMaxVersions as Deprecated.

          Attach a 012 patch and upload to review board, too.
          The new patch add two new SQM: UserScanWithFilterQueryMatcher and RawScanWithFilterQueryMatcher. Fix this bug in these SQM internal. If no filter, it should same with previous implementation. Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - Take a summary about the key points of this fix. 1. Easy to use for user, no need extra work. 2. No more complicated work for ColumnTracker. Only fix this when use filter. 3. Accurate javadoc. And mark scan's setMaxVersions as Deprecated. Attach a 012 patch and upload to review board, too. The new patch add two new SQM: UserScanWithFilterQueryMatcher and RawScanWithFilterQueryMatcher. Fix this bug in these SQM internal. If no filter, it should same with previous implementation. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Attach 013 patch which remove the two new SQM to make code easy to read. And fix the bug in UserScanQueryMatcher internal.

          Show
          zghaobac Guanghao Zhang added a comment - Attach 013 patch which remove the two new SQM to make code easy to read. And fix the bug in UserScanQueryMatcher internal.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 22s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 25s Maven dependency ordering for branch
          +1 mvninstall 7m 13s master passed
          +1 compile 2m 34s master passed
          +1 checkstyle 1m 52s master passed
          +1 mvneclipse 0m 49s master passed
          -1 findbugs 2m 11s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 7m 56s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 1m 59s master passed
          0 mvndep 0m 30s Maven dependency ordering for patch
          +1 mvninstall 2m 51s the patch passed
          +1 compile 2m 36s the patch passed
          +1 javac 2m 36s the patch passed
          +1 checkstyle 1m 50s the patch passed
          +1 mvneclipse 1m 1s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 59m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 8m 15s the patch passed
          -1 javadoc 0m 33s hbase-client generated 8 new + 1 unchanged - 0 fixed = 9 total (was 1)
          +1 unit 3m 42s hbase-client in the patch passed.
          -1 unit 35m 17s hbase-server in the patch failed.
          -1 asflicense 0m 35s The patch generated 3 ASF License warnings.
          144m 10s



          Reason Tests
          Failed junit tests hadoop.hbase.filter.TestFilter
            hadoop.hbase.master.locking.TestLockProcedure
            hadoop.hbase.filter.TestInvocationRecordFilter
            hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan



          Subsystem Report/Notes
          Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874024/HBASE-17125.master.012.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux bfde76c5c6f2 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 7cc458e
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7290/testReport/
          asflicense https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/patch-asflicense-problems.txt
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7290/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 22s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 25s Maven dependency ordering for branch +1 mvninstall 7m 13s master passed +1 compile 2m 34s master passed +1 checkstyle 1m 52s master passed +1 mvneclipse 0m 49s master passed -1 findbugs 2m 11s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 7m 56s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 1m 59s master passed 0 mvndep 0m 30s Maven dependency ordering for patch +1 mvninstall 2m 51s the patch passed +1 compile 2m 36s the patch passed +1 javac 2m 36s the patch passed +1 checkstyle 1m 50s the patch passed +1 mvneclipse 1m 1s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 59m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 8m 15s the patch passed -1 javadoc 0m 33s hbase-client generated 8 new + 1 unchanged - 0 fixed = 9 total (was 1) +1 unit 3m 42s hbase-client in the patch passed. -1 unit 35m 17s hbase-server in the patch failed. -1 asflicense 0m 35s The patch generated 3 ASF License warnings. 144m 10s Reason Tests Failed junit tests hadoop.hbase.filter.TestFilter   hadoop.hbase.master.locking.TestLockProcedure   hadoop.hbase.filter.TestInvocationRecordFilter   hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874024/HBASE-17125.master.012.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux bfde76c5c6f2 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 7cc458e Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7290/testReport/ asflicense https://builds.apache.org/job/PreCommit-HBASE-Build/7290/artifact/patchprocess/patch-asflicense-problems.txt modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7290/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 17m 47s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 32s Maven dependency ordering for branch
          +1 mvninstall 4m 16s master passed
          +1 compile 1m 9s master passed
          +1 checkstyle 0m 53s master passed
          +1 mvneclipse 0m 27s master passed
          -1 findbugs 1m 1s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 2m 44s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 0m 47s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 1s the patch passed
          +1 compile 0m 55s the patch passed
          +1 javac 0m 55s the patch passed
          +1 checkstyle 0m 45s the patch passed
          +1 mvneclipse 0m 24s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 31m 16s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          -1 findbugs 3m 26s hbase-server generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10)
          -1 javadoc 0m 20s hbase-client generated 8 new + 1 unchanged - 0 fixed = 9 total (was 1)
          +1 unit 2m 48s hbase-client in the patch passed.
          -1 unit 22m 1s hbase-server in the patch failed.
          +1 asflicense 0m 16s The patch does not generate ASF License warnings.
          95m 19s



          Reason Tests
          FindBugs module:hbase-server
            Should org.apache.hadoop.hbase.regionserver.querymatcher.UserScanQueryMatcher$SpecifiedNumVersionsColumnFilter be a static inner class? At UserScanQueryMatcher.java:inner class? At UserScanQueryMatcher.java:[lines 272-299]
          Failed junit tests hadoop.hbase.filter.TestFilter
            hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan



          Subsystem Report/Notes
          Docker Client=1.13.1 Server=1.13.1 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874037/HBASE-17125.master.013.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux f36cb54ba435 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 7cc458e
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/new-findbugs-hbase-server.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7291/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7291/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 17m 47s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 32s Maven dependency ordering for branch +1 mvninstall 4m 16s master passed +1 compile 1m 9s master passed +1 checkstyle 0m 53s master passed +1 mvneclipse 0m 27s master passed -1 findbugs 1m 1s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 2m 44s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 0m 47s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 1s the patch passed +1 compile 0m 55s the patch passed +1 javac 0m 55s the patch passed +1 checkstyle 0m 45s the patch passed +1 mvneclipse 0m 24s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 31m 16s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. -1 findbugs 3m 26s hbase-server generated 1 new + 10 unchanged - 0 fixed = 11 total (was 10) -1 javadoc 0m 20s hbase-client generated 8 new + 1 unchanged - 0 fixed = 9 total (was 1) +1 unit 2m 48s hbase-client in the patch passed. -1 unit 22m 1s hbase-server in the patch failed. +1 asflicense 0m 16s The patch does not generate ASF License warnings. 95m 19s Reason Tests FindBugs module:hbase-server   Should org.apache.hadoop.hbase.regionserver.querymatcher.UserScanQueryMatcher$SpecifiedNumVersionsColumnFilter be a static inner class? At UserScanQueryMatcher.java:inner class? At UserScanQueryMatcher.java: [lines 272-299] Failed junit tests hadoop.hbase.filter.TestFilter   hadoop.hbase.io.encoding.TestSeekBeforeWithReverseScan Subsystem Report/Notes Docker Client=1.13.1 Server=1.13.1 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874037/HBASE-17125.master.013.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux f36cb54ba435 4.4.0-43-generic #63-Ubuntu SMP Wed Oct 12 13:48:03 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 7cc458e Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/new-findbugs-hbase-server.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7291/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7291/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7291/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          My aggregate patch v11 has defect in that it doesn't handle ExplicitColumnTracker#reset() correctly.

          ColumnCount should have slack and retract() as well. ExplicitColumnTracker#retract() delegates to ColumnCount#retract().
          When ExplicitColumnTracker#reset() is called, it resets the slack in each ColumnCount to the initial value.

          I have run test suite with updated aggregate patch which passed (other than the tests flagged by flaky test dashboard).

          Show
          yuzhihong@gmail.com Ted Yu added a comment - My aggregate patch v11 has defect in that it doesn't handle ExplicitColumnTracker#reset() correctly. ColumnCount should have slack and retract() as well. ExplicitColumnTracker#retract() delegates to ColumnCount#retract(). When ExplicitColumnTracker#reset() is called, it resets the slack in each ColumnCount to the initial value. I have run test suite with updated aggregate patch which passed (other than the tests flagged by flaky test dashboard).
          Hide
          zghaobac Guanghao Zhang added a comment -

          Ted Yu your "slack" idea is easy to have bug and has side effects. I try it in HBASE-17125.master.no-specified-filter.patch.

          I have run test suite with updated aggregate patch which passed

          As i said in email, "ut passed" doesn't mean that your patch is right... You send me a v11 patch("ut passed"), I pointed the bug in it. Then you fix it. But the latest patch which you send me is still wrong...... Please upload your patch here and review board. And let more people to review to make it right. Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - Ted Yu your "slack" idea is easy to have bug and has side effects. I try it in HBASE-17125 .master.no-specified-filter.patch. I have run test suite with updated aggregate patch which passed As i said in email, "ut passed" doesn't mean that your patch is right... You send me a v11 patch("ut passed"), I pointed the bug in it. Then you fix it. But the latest patch which you send me is still wrong...... Please upload your patch here and review board. And let more people to review to make it right. Thanks.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          17125-slack-13.txt is latest incarnation using slack.

          Currently uploading to review board gave me 'Unknown error'

          Show
          yuzhihong@gmail.com Ted Yu added a comment - 17125-slack-13.txt is latest incarnation using slack. Currently uploading to review board gave me 'Unknown error'
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Review board:
          https://reviews.apache.org/r/60381/

          This is just an alternate approach. Guanghao is the owner of this JIRA.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Review board: https://reviews.apache.org/r/60381/ This is just an alternate approach. Guanghao is the owner of this JIRA.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 13s Docker mode activated.
          0 patch 0m 3s The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for instructions.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 6 new or modified test files.
          0 mvndep 0m 31s Maven dependency ordering for branch
          +1 mvninstall 3m 35s master passed
          +1 compile 1m 12s master passed
          +1 checkstyle 1m 11s master passed
          +1 mvneclipse 0m 36s master passed
          -1 findbugs 0m 57s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 0m 54s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 2m 58s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 1m 7s master passed
          0 mvndep 0m 14s Maven dependency ordering for patch
          +1 mvninstall 1m 6s the patch passed
          +1 compile 0m 55s the patch passed
          +1 javac 0m 55s the patch passed
          +1 checkstyle 0m 48s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 29m 2s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 3m 44s the patch passed
          -1 javadoc 0m 17s hbase-client generated 2 new + 1 unchanged - 1 fixed = 3 total (was 2)
          +1 unit 2m 24s hbase-client in the patch passed.
          -1 unit 117m 5s hbase-server in the patch failed.
          +1 asflicense 0m 41s The patch does not generate ASF License warnings.
          171m 11s



          Reason Tests
          Failed junit tests hadoop.hbase.master.procedure.TestMasterProcedureWalLease



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874152/17125-slack-13.txt
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 18be69b73ea7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 7cc458e
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7295/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7295/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 13s Docker mode activated. 0 patch 0m 3s The patch file was not named according to hbase's naming conventions. Please see https://yetus.apache.org/documentation/0.3.0/precommit-patchnames for instructions. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 6 new or modified test files. 0 mvndep 0m 31s Maven dependency ordering for branch +1 mvninstall 3m 35s master passed +1 compile 1m 12s master passed +1 checkstyle 1m 11s master passed +1 mvneclipse 0m 36s master passed -1 findbugs 0m 57s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 0m 54s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 2m 58s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 1m 7s master passed 0 mvndep 0m 14s Maven dependency ordering for patch +1 mvninstall 1m 6s the patch passed +1 compile 0m 55s the patch passed +1 javac 0m 55s the patch passed +1 checkstyle 0m 48s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 29m 2s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 3m 44s the patch passed -1 javadoc 0m 17s hbase-client generated 2 new + 1 unchanged - 1 fixed = 3 total (was 2) +1 unit 2m 24s hbase-client in the patch passed. -1 unit 117m 5s hbase-server in the patch failed. +1 asflicense 0m 41s The patch does not generate ASF License warnings. 171m 11s Reason Tests Failed junit tests hadoop.hbase.master.procedure.TestMasterProcedureWalLease Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874152/17125-slack-13.txt JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 18be69b73ea7 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 7cc458e Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7295/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7295/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7295/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Reviewing HBASE-17125.master.014.patch now

          Show
          anoop.hbase Anoop Sam John added a comment - Reviewing HBASE-17125 .master.014.patch now
          Hide
          anoop.hbase Anoop Sam John added a comment -

          So this patch is in line with some old proposal that u put right.. I like it.. Doing some more careful reviews. Pls put the latest patch in RB

          Show
          anoop.hbase Anoop Sam John added a comment - So this patch is in line with some old proposal that u put right.. I like it.. Doing some more careful reviews. Pls put the latest patch in RB
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 32m 38s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 38s Maven dependency ordering for branch
          +1 mvninstall 8m 21s master passed
          +1 compile 3m 20s master passed
          +1 checkstyle 2m 49s master passed
          +1 mvneclipse 1m 31s master passed
          -1 findbugs 2m 36s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 2m 47s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 7m 31s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 2m 47s master passed
          0 mvndep 0m 34s Maven dependency ordering for patch
          +1 mvninstall 2m 49s the patch passed
          +1 compile 2m 38s the patch passed
          +1 javac 2m 38s the patch passed
          +1 checkstyle 1m 55s the patch passed
          +1 mvneclipse 1m 6s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 69m 6s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 8m 57s the patch passed
          -1 javadoc 0m 52s hbase-client generated 8 new + 1 unchanged - 1 fixed = 9 total (was 2)
          +1 unit 4m 48s hbase-client in the patch passed.
          -1 unit 46m 35s hbase-server in the patch failed.
          +1 asflicense 0m 40s The patch does not generate ASF License warnings.
          207m 48s



          Reason Tests
          Failed junit tests hadoop.hbase.filter.TestFilter



          Subsystem Report/Notes
          Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874200/HBASE-17125.master.014.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux b7c5a776da35 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh
          git revision master / d092008
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7299/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7299/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 32m 38s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 38s Maven dependency ordering for branch +1 mvninstall 8m 21s master passed +1 compile 3m 20s master passed +1 checkstyle 2m 49s master passed +1 mvneclipse 1m 31s master passed -1 findbugs 2m 36s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 2m 47s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 7m 31s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 2m 47s master passed 0 mvndep 0m 34s Maven dependency ordering for patch +1 mvninstall 2m 49s the patch passed +1 compile 2m 38s the patch passed +1 javac 2m 38s the patch passed +1 checkstyle 1m 55s the patch passed +1 mvneclipse 1m 6s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 69m 6s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 8m 57s the patch passed -1 javadoc 0m 52s hbase-client generated 8 new + 1 unchanged - 1 fixed = 9 total (was 2) +1 unit 4m 48s hbase-client in the patch passed. -1 unit 46m 35s hbase-server in the patch failed. +1 asflicense 0m 40s The patch does not generate ASF License warnings. 207m 48s Reason Tests Failed junit tests hadoop.hbase.filter.TestFilter Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874200/HBASE-17125.master.014.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux b7c5a776da35 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build@2/component/dev-support/hbase-personality.sh git revision master / d092008 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7299/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7299/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7299/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Based on patch v16:

          testColumnPaginationFilter(org.apache.hadoop.hbase.filter.TestFilter)  Time elapsed: 0.165 sec  <<< FAILURE!
          java.lang.AssertionError
          	at org.apache.hadoop.hbase.filter.TestFilter.verifyScan(TestFilter.java:1667)
          	at org.apache.hadoop.hbase.filter.TestFilter.testColumnPaginationFilter(TestFilter.java:1945)
          ...
          testGet_Basic(org.apache.hadoop.hbase.regionserver.TestHRegion)  Time elapsed: 0.099 sec  <<< FAILURE!
          java.lang.AssertionError
          	at org.apache.hadoop.hbase.regionserver.TestHRegion.testGet_Basic(TestHRegion.java:2616)
          

          Can you double check ?

          Thanks

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Based on patch v16: testColumnPaginationFilter(org.apache.hadoop.hbase.filter.TestFilter) Time elapsed: 0.165 sec <<< FAILURE! java.lang.AssertionError at org.apache.hadoop.hbase.filter.TestFilter.verifyScan(TestFilter.java:1667) at org.apache.hadoop.hbase.filter.TestFilter.testColumnPaginationFilter(TestFilter.java:1945) ... testGet_Basic(org.apache.hadoop.hbase.regionserver.TestHRegion) Time elapsed: 0.099 sec <<< FAILURE! java.lang.AssertionError at org.apache.hadoop.hbase.regionserver.TestHRegion.testGet_Basic(TestHRegion.java:2616) Can you double check ? Thanks
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 36s Maven dependency ordering for branch
          +1 mvninstall 6m 50s master passed
          +1 compile 2m 33s master passed
          +1 checkstyle 2m 13s master passed
          +1 mvneclipse 1m 1s master passed
          -1 findbugs 1m 56s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 1m 52s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 6m 31s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 2m 5s master passed
          0 mvndep 0m 26s Maven dependency ordering for patch
          +1 mvninstall 2m 11s the patch passed
          +1 compile 2m 8s the patch passed
          +1 javac 2m 8s the patch passed
          +1 checkstyle 1m 45s the patch passed
          +1 mvneclipse 0m 55s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 58m 15s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 7m 47s the patch passed
          -1 javadoc 0m 33s hbase-client generated 5 new + 1 unchanged - 1 fixed = 6 total (was 2)
          +1 unit 3m 40s hbase-client in the patch passed.
          -1 unit 36m 12s hbase-server in the patch failed.
          +1 asflicense 0m 34s The patch does not generate ASF License warnings.
          142m 30s



          Reason Tests
          Failed junit tests hadoop.hbase.filter.TestFilter



          Subsystem Report/Notes
          Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874342/HBASE-17125.master.016.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 84ee331aa840 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / d092008
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7310/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7310/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 36s Maven dependency ordering for branch +1 mvninstall 6m 50s master passed +1 compile 2m 33s master passed +1 checkstyle 2m 13s master passed +1 mvneclipse 1m 1s master passed -1 findbugs 1m 56s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 1m 52s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 6m 31s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 2m 5s master passed 0 mvndep 0m 26s Maven dependency ordering for patch +1 mvninstall 2m 11s the patch passed +1 compile 2m 8s the patch passed +1 javac 2m 8s the patch passed +1 checkstyle 1m 45s the patch passed +1 mvneclipse 0m 55s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 58m 15s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 7m 47s the patch passed -1 javadoc 0m 33s hbase-client generated 5 new + 1 unchanged - 1 fixed = 6 total (was 2) +1 unit 3m 40s hbase-client in the patch passed. -1 unit 36m 12s hbase-server in the patch failed. +1 asflicense 0m 34s The patch does not generate ASF License warnings. 142m 30s Reason Tests Failed junit tests hadoop.hbase.filter.TestFilter Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874342/HBASE-17125.master.016.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 84ee331aa840 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / d092008 Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7310/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7310/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7310/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Attach 017 patch which fixed the failed ut.

          Show
          zghaobac Guanghao Zhang added a comment - Attach 017 patch which fixed the failed ut.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 23s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 26s Maven dependency ordering for branch
          +1 mvninstall 6m 36s master passed
          +1 compile 1m 57s master passed
          +1 checkstyle 1m 28s master passed
          +1 mvneclipse 0m 44s master passed
          -1 findbugs 1m 50s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 5m 54s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 1m 30s master passed
          0 mvndep 0m 25s Maven dependency ordering for patch
          +1 mvninstall 2m 12s the patch passed
          +1 compile 2m 2s the patch passed
          +1 javac 2m 2s the patch passed
          +1 checkstyle 1m 28s the patch passed
          +1 mvneclipse 0m 43s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 58m 43s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 8m 2s the patch passed
          -1 javadoc 0m 33s hbase-client generated 5 new + 1 unchanged - 0 fixed = 6 total (was 1)
          +1 unit 3m 41s hbase-client in the patch passed.
          -1 unit 33m 54s hbase-server in the patch failed.
          +1 asflicense 0m 26s The patch does not generate ASF License warnings.
          134m 51s



          Reason Tests
          Failed junit tests hadoop.hbase.master.locking.TestLockProcedure



          Subsystem Report/Notes
          Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874373/HBASE-17125.master.017.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 4c11bb685b75 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 96aca6b
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7313/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7313/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 23s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 26s Maven dependency ordering for branch +1 mvninstall 6m 36s master passed +1 compile 1m 57s master passed +1 checkstyle 1m 28s master passed +1 mvneclipse 0m 44s master passed -1 findbugs 1m 50s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 5m 54s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 1m 30s master passed 0 mvndep 0m 25s Maven dependency ordering for patch +1 mvninstall 2m 12s the patch passed +1 compile 2m 2s the patch passed +1 javac 2m 2s the patch passed +1 checkstyle 1m 28s the patch passed +1 mvneclipse 0m 43s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 58m 43s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 8m 2s the patch passed -1 javadoc 0m 33s hbase-client generated 5 new + 1 unchanged - 0 fixed = 6 total (was 1) +1 unit 3m 41s hbase-client in the patch passed. -1 unit 33m 54s hbase-server in the patch failed. +1 asflicense 0m 26s The patch does not generate ASF License warnings. 134m 51s Reason Tests Failed junit tests hadoop.hbase.master.locking.TestLockProcedure Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874373/HBASE-17125.master.017.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 4c11bb685b75 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 96aca6b Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7313/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7313/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7313/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment - - edited
          +public class ColumnTrackerWrapper implements ColumnTracker {
          

          There may be more wrapper for ColumnTracker in the future.
          How about calling this wrapper FilterAwareColumnTracker (or something like that) ?

          Show
          yuzhihong@gmail.com Ted Yu added a comment - - edited + public class ColumnTrackerWrapper implements ColumnTracker { There may be more wrapper for ColumnTracker in the future. How about calling this wrapper FilterAwareColumnTracker (or something like that) ?
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -
          +    final WAL wal = HBaseTestingUtility.createWal(TEST_UTIL.getConfiguration(), logDir, info);
          +    this.region = TEST_UTIL.createLocalHRegion(info, htd, wal);
          

          The above code relies on other test to initialize chunk creator.
          If you run the subtest alone, you would observe NPE like the following:

          MemStoreLABImpl.getOrMakeChunk() line: 242
          MemStoreLABImpl.copyCellInto(Cell) line: 118
          MutableSegment(Segment).maybeCloneWithAllocator(Cell) line: 168
          CompactingMemStore(AbstractMemStore).maybeCloneWithAllocator(Cell) line: 268
          CompactingMemStore(AbstractMemStore).add(Cell, MemstoreSize) line: 107
          CompactingMemStore(AbstractMemStore).add(Iterable<Cell>, MemstoreSize) line: 101
          HStore.add(Iterable<Cell>, MemstoreSize) line: 711
          HRegion.applyToMemstore(Store, List<Cell>, boolean, MemstoreSize) line: 4001
          HRegion.applyFamilyMapToMemstore(Map<byte[],List<Cell>>, MemstoreSize) line: 3984
          HRegion.doMiniBatchMutate(BatchOperation<?>) line: 3439
          HRegion.batchMutate(BatchOperation<?>) line: 3131
          HRegion.batchMutate(Mutation[], long, long) line: 3073
          HRegion.batchMutate(Mutation[]) line: 3077
          HRegion.doBatchMutate(Mutation) line: 3827
          HRegion.put(Put) line: 2950
          TestHRegion.testGetWithFilter() line: 2665
          

          The following would allow the subtest to run alone:

          +    ChunkCreator.initialize(MemStoreLABImpl.CHUNK_SIZE_DEFAULT, false, 0, 0, 0, null);
          +    this.region = TEST_UTIL.createLocalHRegion(info, htd, wal);
          
          Show
          yuzhihong@gmail.com Ted Yu added a comment - + final WAL wal = HBaseTestingUtility.createWal(TEST_UTIL.getConfiguration(), logDir, info); + this .region = TEST_UTIL.createLocalHRegion(info, htd, wal); The above code relies on other test to initialize chunk creator. If you run the subtest alone, you would observe NPE like the following: MemStoreLABImpl.getOrMakeChunk() line: 242 MemStoreLABImpl.copyCellInto(Cell) line: 118 MutableSegment(Segment).maybeCloneWithAllocator(Cell) line: 168 CompactingMemStore(AbstractMemStore).maybeCloneWithAllocator(Cell) line: 268 CompactingMemStore(AbstractMemStore).add(Cell, MemstoreSize) line: 107 CompactingMemStore(AbstractMemStore).add(Iterable<Cell>, MemstoreSize) line: 101 HStore.add(Iterable<Cell>, MemstoreSize) line: 711 HRegion.applyToMemstore(Store, List<Cell>, boolean , MemstoreSize) line: 4001 HRegion.applyFamilyMapToMemstore(Map< byte [],List<Cell>>, MemstoreSize) line: 3984 HRegion.doMiniBatchMutate(BatchOperation<?>) line: 3439 HRegion.batchMutate(BatchOperation<?>) line: 3131 HRegion.batchMutate(Mutation[], long , long ) line: 3073 HRegion.batchMutate(Mutation[]) line: 3077 HRegion.doBatchMutate(Mutation) line: 3827 HRegion.put(Put) line: 2950 TestHRegion.testGetWithFilter() line: 2665 The following would allow the subtest to run alone: + ChunkCreator.initialize(MemStoreLABImpl.CHUNK_SIZE_DEFAULT, false , 0, 0, 0, null ); + this .region = TEST_UTIL.createLocalHRegion(info, htd, wal);
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Since existing unit tests didn't uncover the defect in 17125-slack-13.txt, please add new test to show that patch v17 is correct in this regard.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Since existing unit tests didn't uncover the defect in 17125-slack-13.txt, please add new test to show that patch v17 is correct in this regard.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          "FilterAwareColumnTracker " - Sounds good.
          Ya +1 on adding more tests on diff scenarios. Thanks

          Show
          anoop.hbase Anoop Sam John added a comment - "FilterAwareColumnTracker " - Sounds good. Ya +1 on adding more tests on diff scenarios. Thanks
          Hide
          zghaobac Guanghao Zhang added a comment -

          Attach a 018 patch.

          The following would allow the subtest to run alone:

          Fixed it.

          How about calling this wrapper FilterAwareColumnTracker (or something like that) ?

          Refactored.

          adding more tests on diff scenarios.

          The ut is TestFromClientSide#testReadWithFilter and TestHRegion#testGetWithFilter.

          Show
          zghaobac Guanghao Zhang added a comment - Attach a 018 patch. The following would allow the subtest to run alone: Fixed it. How about calling this wrapper FilterAwareColumnTracker (or something like that) ? Refactored. adding more tests on diff scenarios. The ut is TestFromClientSide#testReadWithFilter and TestHRegion#testGetWithFilter.
          Hide
          Apache9 Duo Zhang added a comment -

          Can we avoid changing the ColumnTracker interface? If you all really hate modifying the filter of SQM, then my bottom line is to implement the logic of FilterAwareColumnTracker inside SQM directly.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Can we avoid changing the ColumnTracker interface? If you all really hate modifying the filter of SQM, then my bottom line is to implement the logic of FilterAwareColumnTracker inside SQM directly. Thanks.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          The ut is TestFromClientSide#testReadWithFilter and TestHRegion#testGetWithFilter

          I plugged in these two tests into 17125-slack-13.txt.
          They passed.

          Please double check.

          Show
          yuzhihong@gmail.com Ted Yu added a comment - The ut is TestFromClientSide#testReadWithFilter and TestHRegion#testGetWithFilter I plugged in these two tests into 17125-slack-13.txt. They passed. Please double check.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 hbaseanti 0m 1s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 3 new or modified test files.
          0 mvndep 0m 27s Maven dependency ordering for branch
          +1 mvninstall 3m 47s master passed
          +1 compile 1m 23s master passed
          +1 checkstyle 1m 24s master passed
          +1 mvneclipse 0m 38s master passed
          -1 findbugs 0m 58s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 0m 58s hbase-client in master has 4 extant Findbugs warnings.
          -1 findbugs 3m 8s hbase-server in master has 10 extant Findbugs warnings.
          +1 javadoc 1m 11s master passed
          0 mvndep 0m 16s Maven dependency ordering for patch
          +1 mvninstall 1m 12s the patch passed
          +1 compile 1m 2s the patch passed
          +1 javac 1m 2s the patch passed
          +1 checkstyle 0m 51s the patch passed
          +1 mvneclipse 0m 30s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 34m 24s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3.
          +1 findbugs 4m 45s the patch passed
          -1 javadoc 0m 21s hbase-client generated 5 new + 1 unchanged - 1 fixed = 6 total (was 2)
          +1 unit 2m 44s hbase-client in the patch passed.
          -1 unit 118m 33s hbase-server in the patch failed.
          +1 asflicense 0m 57s The patch does not generate ASF License warnings.
          181m 7s



          Reason Tests
          Failed junit tests hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput
          Timed out junit tests org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2
            org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer
            org.apache.hadoop.hbase.master.TestDistributedLogSplitting
            org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874472/HBASE-17125.master.018.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 840e53c1fff1 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 2d781aa
          Default Java 1.8.0_131
          findbugs v3.1.0-RC1
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html
          javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7324/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7324/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 hbaseanti 0m 1s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 3 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 3m 47s master passed +1 compile 1m 23s master passed +1 checkstyle 1m 24s master passed +1 mvneclipse 0m 38s master passed -1 findbugs 0m 58s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 0m 58s hbase-client in master has 4 extant Findbugs warnings. -1 findbugs 3m 8s hbase-server in master has 10 extant Findbugs warnings. +1 javadoc 1m 11s master passed 0 mvndep 0m 16s Maven dependency ordering for patch +1 mvninstall 1m 12s the patch passed +1 compile 1m 2s the patch passed +1 javac 1m 2s the patch passed +1 checkstyle 0m 51s the patch passed +1 mvneclipse 0m 30s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 34m 24s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha3. +1 findbugs 4m 45s the patch passed -1 javadoc 0m 21s hbase-client generated 5 new + 1 unchanged - 1 fixed = 6 total (was 2) +1 unit 2m 44s hbase-client in the patch passed. -1 unit 118m 33s hbase-server in the patch failed. +1 asflicense 0m 57s The patch does not generate ASF License warnings. 181m 7s Reason Tests Failed junit tests hadoop.hbase.io.asyncfs.TestSaslFanOutOneBlockAsyncDFSOutput Timed out junit tests org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer2   org.apache.hadoop.hbase.master.balancer.TestStochasticLoadBalancer   org.apache.hadoop.hbase.master.TestDistributedLogSplitting   org.apache.hadoop.hbase.snapshot.TestExportSnapshotNoCluster Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:757bf37 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12874472/HBASE-17125.master.018.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 840e53c1fff1 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 2d781aa Default Java 1.8.0_131 findbugs v3.1.0-RC1 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/branch-findbugs-hbase-client-warnings.html findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/branch-findbugs-hbase-server-warnings.html javadoc https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/diff-javadoc-javadoc-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/7324/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/7324/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/7324/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.

            People

            • Assignee:
              zghaobac Guanghao Zhang
              Reporter:
              zghaobac Guanghao Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:

                Development