Uploaded image for project: 'HBase'
  1. HBase
  2. HBASE-17125

Inconsistent result when use filter to read data

    Details

    • Type: Bug
    • Status: Patch Available
    • Priority: Critical
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 2.0.0
    • Component/s: None
    • Labels:
      None

      Description

      Assume a cloumn's max versions is 3, then we write 4 versions of this column. The oldest version doesn't remove immediately. But from the user view, the oldest version has gone. When user use a filter to query, if the filter skip a new version, then the oldest version will be seen again. But after compact the region, then the oldest version will never been seen. So it is weird for user. The query will get inconsistent result before and after region compaction.

      The reason is matchColumn method of UserScanQueryMatcher. It first check the cell by filter, then check the number of versions needed. So if the filter skip the new version, then the oldest version will be seen again when it is not removed.

      Have a discussion offline with Duo Zhang and Honghua Feng, now we have two solution for this problem. The first idea is check the number of versions first, then check the cell by filter. As the comment of setFilter, the filter is called after all tests for ttl, column match, deletes and max versions have been run.

        /**
         * Apply the specified server-side filter when performing the Query.
         * Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests
         * for ttl, column match, deletes and max versions have been run.
         * @param filter filter to run on the server
         * @return this for invocation chaining
         */
        public Query setFilter(Filter filter) {
          this.filter = filter;
          return this;
        }
      

      But this idea has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore.

      So the second idea has three steps.
      1. check by the max versions of this column
      2. check the kv by filter
      3. check the versions which user need.
      But this will lead the ScanQueryMatcher more complicated. And this will break the javadoc of Query.setFilter.

      Now we don't have a final solution for this problem. Suggestions are welcomed.

      1. example.diff
        4 kB
        Guanghao Zhang
      2. HBASE-17125.master.001.patch
        12 kB
        Guanghao Zhang
      3. HBASE-17125.master.002.patch
        13 kB
        Guanghao Zhang
      4. HBASE-17125.master.002.patch
        13 kB
        Guanghao Zhang
      5. HBASE-17125.master.003.patch
        28 kB
        Guanghao Zhang
      6. HBASE-17125.master.004.patch
        29 kB
        Guanghao Zhang
      7. HBASE-17125.master.005.patch
        29 kB
        Guanghao Zhang
      8. HBASE-17125.master.006.patch
        30 kB
        Guanghao Zhang
      9. HBASE-17125.master.007.patch
        62 kB
        Guanghao Zhang
      10. HBASE-17125.master.008.patch
        62 kB
        Guanghao Zhang
      11. HBASE-17125.master.009.patch
        66 kB
        Guanghao Zhang
      12. HBASE-17125.master.009.patch
        65 kB
        Guanghao Zhang

        Issue Links

          Activity

          Hide
          zghaobac Guanghao Zhang added a comment -

          Upload a test example for this problem.

          Show
          zghaobac Guanghao Zhang added a comment - Upload a test example for this problem.
          Hide
          Apache9 Duo Zhang added a comment -

          I prefer the simple 'check versions then check filter' fix, but I'm not sure if some daughter projects need the old behavior.

          Andrew Purtell Does Phoenix need the old behavior where maxVersions is a limit for the whole result, not the result before filtering?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - I prefer the simple 'check versions then check filter' fix, but I'm not sure if some daughter projects need the old behavior. Andrew Purtell Does Phoenix need the old behavior where maxVersions is a limit for the whole result, not the result before filtering? Thanks.
          Hide
          apurtell Andrew Purtell added a comment -
          Show
          apurtell Andrew Purtell added a comment - James Taylor
          Hide
          Apache9 Duo Zhang added a comment -
          Show
          Apache9 Duo Zhang added a comment - Ping James Taylor .
          Hide
          apurtell Andrew Purtell added a comment -

          Let's wait a week due to the thanksgiving holiday in the US, then proceed as you like if still no response. We can run the Phoenix unit test suite with a patched HBase to check if they are obviously affected.

          Show
          apurtell Andrew Purtell added a comment - Let's wait a week due to the thanksgiving holiday in the US, then proceed as you like if still no response. We can run the Phoenix unit test suite with a patched HBase to check if they are obviously affected.
          Hide
          Apache9 Duo Zhang added a comment -

          Fine. Thanks.

          Show
          Apache9 Duo Zhang added a comment - Fine. Thanks.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          I think this was discussed previouly also. Lars Hofhansl had some similar thoughts. But I think not only down stream projects but many user apps may be running with the current logic. If we change it then it could have a bigger impact.
          May be we should introduce the new one as a toggle or by allowing user to configure this new ScanTracker? Am not sure on the best soln but just saying.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - I think this was discussed previouly also. Lars Hofhansl had some similar thoughts. But I think not only down stream projects but many user apps may be running with the current logic. If we change it then it could have a bigger impact. May be we should introduce the new one as a toggle or by allowing user to configure this new ScanTracker? Am not sure on the best soln but just saying.
          Hide
          Apache9 Duo Zhang added a comment -

          I think we can do it for 2.0 only.

          The old logic is not stable, I do not think we should let user depend on a broken semantic anymore, especially that we never guarantee this in the documentation.

          Show
          Apache9 Duo Zhang added a comment - I think we can do it for 2.0 only. The old logic is not stable, I do not think we should let user depend on a broken semantic anymore, especially that we never guarantee this in the documentation.
          Hide
          lhofhansl Lars Hofhansl added a comment - - edited

          Simply put: There is no right order in which to evaluate filters, versions, column tracking, and deletes. Regardless of which order you pick someone will complain about it.

          This has changed various times in the past. I think we should stick with how it is. If filters can see versions they can act accordingly (and filter extra versions if needed). If filters do not see all version, but you happen to need to look at all versions... Well then you'd be out of luck if we make this change.

          See also HBASE-13094. That one is about the order of filters w.r.t. delete markers.

          I guess, in hindsight for maximum flexibility filters should have been placed before everything else so that they can be used to intervene into the entire pipeline.

          Show
          lhofhansl Lars Hofhansl added a comment - - edited Simply put: There is no right order in which to evaluate filters, versions, column tracking, and deletes. Regardless of which order you pick someone will complain about it. This has changed various times in the past. I think we should stick with how it is. If filters can see versions they can act accordingly (and filter extra versions if needed). If filters do not see all version, but you happen to need to look at all versions... Well then you'd be out of luck if we make this change. See also HBASE-13094 . That one is about the order of filters w.r.t. delete markers. I guess, in hindsight for maximum flexibility filters should have been placed before everything else so that they can be used to intervene into the entire pipeline.
          Hide
          Apache9 Duo Zhang added a comment -

          HBASE-13094 aims to add a new type of filter right? This issue aims to solve the problem for the current fiter implementation.

          And I need to speak again, the old 'filter then check verions' is not stable. The same request can return different results only if there is a major compaction. This is a bug and we need to fix it.

          And the biggest problem here, is the semantic of setMaxVerions. In the javadoc of setFilter method, the max versions check should be evaluated before filtering but the current implementation is more likely a limitation of the final result. This leads to two approaches to fix the bug, one is try to break our javadoc and the new implementation will be more complicated, but can keep compatible with the old implementation as much as possible. The other is to change the implementation according to our javadoc, the fix is simple and straight-forward.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - HBASE-13094 aims to add a new type of filter right? This issue aims to solve the problem for the current fiter implementation. And I need to speak again, the old 'filter then check verions' is not stable. The same request can return different results only if there is a major compaction. This is a bug and we need to fix it. And the biggest problem here, is the semantic of setMaxVerions. In the javadoc of setFilter method, the max versions check should be evaluated before filtering but the current implementation is more likely a limitation of the final result. This leads to two approaches to fix the bug, one is try to break our javadoc and the new implementation will be more complicated, but can keep compatible with the old implementation as much as possible. The other is to change the implementation according to our javadoc, the fix is simple and straight-forward. Thanks.
          Hide
          huaxiang huaxiang sun added a comment -

          Hi Guanghao Zhang, "But this idea has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore." I think this is caused by the different meaning of scan.maxVersions and HCD's maxVersions? The implementation implies that they are same. Thanks.

          From doc:
          Scan setMaxVersions(int maxVersions)
          Get up to the specified number of versions of each column.

          The implementation:
          https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/UserScanQueryMatcher.java#L197

          Show
          huaxiang huaxiang sun added a comment - Hi Guanghao Zhang , "But this idea has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore." I think this is caused by the different meaning of scan.maxVersions and HCD's maxVersions? The implementation implies that they are same. Thanks. From doc: Scan setMaxVersions(int maxVersions) Get up to the specified number of versions of each column. The implementation: https://github.com/apache/hbase/blob/master/hbase-server/src/main/java/org/apache/hadoop/hbase/regionserver/querymatcher/UserScanQueryMatcher.java#L197
          Hide
          stack stack added a comment -

          Marking as critical for 2.0. These 'unexpected behaviors' cause operator head-scratching and wasted hours of digging.

          Show
          stack stack added a comment - Marking as critical for 2.0. These 'unexpected behaviors' cause operator head-scratching and wasted hours of digging.
          Hide
          zghaobac Guanghao Zhang added a comment -

          I thought this should be fixed by the second idea.
          1. check by the max versions of this column, HCD's maxVersions
          2. check the kv by filter
          3. check the versions which user need, scan.maxVersions

          huaxiang sun Stack Andrew Purtell Lars Hofhansl ramkrishna.s.vasudevan Duo Zhang What do you think about this idea?

          Show
          zghaobac Guanghao Zhang added a comment - I thought this should be fixed by the second idea. 1. check by the max versions of this column, HCD's maxVersions 2. check the kv by filter 3. check the versions which user need, scan.maxVersions huaxiang sun Stack Andrew Purtell Lars Hofhansl ramkrishna.s.vasudevan Duo Zhang What do you think about this idea?
          Hide
          Apache9 Duo Zhang added a comment -

          I still think the simple fix is enough. 'Filter' is designed to move the computation from client to server, which implicitly means you can also implement the logic at client side. I think that's why our document says filter is tested at last, after ttl, column match, deletes and max versions.

          And I think we can introduce a new filter to provide the function that return specific versions after filter? Maybe called 'MaxVersionFilter'? You can set maxVersions to infinite and put MaxVersionFilter at the last position in FilterList to limit the versions returned to client.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - I still think the simple fix is enough. 'Filter' is designed to move the computation from client to server, which implicitly means you can also implement the logic at client side. I think that's why our document says filter is tested at last, after ttl, column match, deletes and max versions. And I think we can introduce a new filter to provide the function that return specific versions after filter? Maybe called 'MaxVersionFilter'? You can set maxVersions to infinite and put MaxVersionFilter at the last position in FilterList to limit the versions returned to client. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Upload a v1 patch. Discussed offline with Duo Zhang, we plan use the first simple idea to fix it. The ScanQueryMatcher will check the number of versions first, then check the cell by filter. This fix doesn't break the javadoc of Query.setFilter.

            /**
             * Apply the specified server-side filter when performing the Query.
             * Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests
             * for ttl, column match, deletes and max versions have been run.
             * @param filter filter to run on the server
             * @return this for invocation chaining
             */
            public Query setFilter(Filter filter) {
              this.filter = filter;
              return this;
            }
          
          Show
          zghaobac Guanghao Zhang added a comment - Upload a v1 patch. Discussed offline with Duo Zhang , we plan use the first simple idea to fix it. The ScanQueryMatcher will check the number of versions first, then check the cell by filter. This fix doesn't break the javadoc of Query.setFilter. /** * Apply the specified server-side filter when performing the Query. * Only {@link Filter#filterKeyValue(Cell)} is called AFTER all tests * for ttl, column match, deletes and max versions have been run. * @param filter filter to run on the server * @ return this for invocation chaining */ public Query setFilter(Filter filter) { this .filter = filter; return this ; }
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 15s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 3m 6s master passed
          +1 compile 0m 36s master passed
          +1 checkstyle 0m 46s master passed
          +1 mvneclipse 0m 15s master passed
          +1 findbugs 1m 53s master passed
          +1 javadoc 0m 29s master passed
          +1 mvninstall 0m 41s the patch passed
          +1 compile 0m 40s the patch passed
          +1 javac 0m 40s the patch passed
          +1 checkstyle 0m 47s the patch passed
          +1 mvneclipse 0m 15s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 29m 21s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 11s the patch passed
          +1 javadoc 0m 28s the patch passed
          -1 unit 23m 23s hbase-server in the patch failed.
          +1 asflicense 0m 11s The patch does not generate ASF License warnings.
          65m 37s



          Reason Tests
          Failed junit tests hadoop.hbase.regionserver.TestMinVersions



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856950/HBASE-17125.master.001.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 3217dd7d6291 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 1160315
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6028/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6028/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 15s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 3m 6s master passed +1 compile 0m 36s master passed +1 checkstyle 0m 46s master passed +1 mvneclipse 0m 15s master passed +1 findbugs 1m 53s master passed +1 javadoc 0m 29s master passed +1 mvninstall 0m 41s the patch passed +1 compile 0m 40s the patch passed +1 javac 0m 40s the patch passed +1 checkstyle 0m 47s the patch passed +1 mvneclipse 0m 15s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 29m 21s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 11s the patch passed +1 javadoc 0m 28s the patch passed -1 unit 23m 23s hbase-server in the patch failed. +1 asflicense 0m 11s The patch does not generate ASF License warnings. 65m 37s Reason Tests Failed junit tests hadoop.hbase.regionserver.TestMinVersions Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12856950/HBASE-17125.master.001.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 3217dd7d6291 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 1160315 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6028/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6028/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6028/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Try to trigger Hadoop QA again.

          Show
          zghaobac Guanghao Zhang added a comment - Try to trigger Hadoop QA again.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 2 new or modified test files.
          +1 mvninstall 4m 24s master passed
          +1 compile 0m 46s master passed
          +1 checkstyle 0m 56s master passed
          +1 mvneclipse 0m 18s master passed
          +1 findbugs 2m 11s master passed
          +1 javadoc 0m 39s master passed
          +1 mvninstall 0m 55s the patch passed
          +1 compile 0m 47s the patch passed
          +1 javac 0m 47s the patch passed
          +1 checkstyle 0m 54s the patch passed
          +1 mvneclipse 0m 18s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 33m 13s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 6s the patch passed
          +1 javadoc 0m 37s the patch passed
          -1 unit 108m 25s hbase-server in the patch failed.
          +1 asflicense 0m 21s The patch does not generate ASF License warnings.
          157m 38s



          Reason Tests
          Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes
            hadoop.hbase.security.visibility.TestVisibilityLabelsWithCustomVisLabService
            hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService



          Subsystem Report/Notes
          Docker Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12857288/HBASE-17125.master.002.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux dc1065f7362a 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / e0365df
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6053/testReport/
          modules C: hbase-server U: hbase-server
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6053/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 2 new or modified test files. +1 mvninstall 4m 24s master passed +1 compile 0m 46s master passed +1 checkstyle 0m 56s master passed +1 mvneclipse 0m 18s master passed +1 findbugs 2m 11s master passed +1 javadoc 0m 39s master passed +1 mvninstall 0m 55s the patch passed +1 compile 0m 47s the patch passed +1 javac 0m 47s the patch passed +1 checkstyle 0m 54s the patch passed +1 mvneclipse 0m 18s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 33m 13s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 6s the patch passed +1 javadoc 0m 37s the patch passed -1 unit 108m 25s hbase-server in the patch failed. +1 asflicense 0m 21s The patch does not generate ASF License warnings. 157m 38s Reason Tests Failed junit tests hadoop.hbase.security.visibility.TestVisibilityLabelsWithDeletes   hadoop.hbase.security.visibility.TestVisibilityLabelsWithCustomVisLabService   hadoop.hbase.security.visibility.TestVisibilityLabelsWithDefaultVisLabelService Subsystem Report/Notes Docker Client=1.11.2 Server=1.11.2 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12857288/HBASE-17125.master.002.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux dc1065f7362a 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / e0365df Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6053/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6053/testReport/ modules C: hbase-server U: hbase-server Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6053/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          huaxiang huaxiang sun added a comment -

          Looks good to me. Maybe update the description of Scan and Get 's setMaxVersions() to make sure there is no confusion?

            /**
             * Get up to the specified number of versions of each column. 
              // Something like "set the maximum versions of each column the scan will check"?
             * @param maxVersions maximum versions for each column
             * @return this
             */
            public Scan setMaxVersions(int maxVersions) {
              this.maxVersions = maxVersions;
              return this;
            }
          
          
          Show
          huaxiang huaxiang sun added a comment - Looks good to me. Maybe update the description of Scan and Get 's setMaxVersions() to make sure there is no confusion? /** * Get up to the specified number of versions of each column. // Something like "set the maximum versions of each column the scan will check" ? * @param maxVersions maximum versions for each column * @ return this */ public Scan setMaxVersions( int maxVersions) { this .maxVersions = maxVersions; return this ; }
          Hide
          zghaobac Guanghao Zhang added a comment -

          huaxiang sun Thanks for point out this. The new implemention will break setMaxVersions's javadoc. I will update it in next patch.

          Show
          zghaobac Guanghao Zhang added a comment - huaxiang sun Thanks for point out this. The new implemention will break setMaxVersions's javadoc. I will update it in next patch.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Failed ut TestVisibilityLabelsWithDeletes is related.

          The reason is about the Delete the latest version of the specified column.

            /**
             * Delete the latest version of the specified column.
             * This is an expensive call in that on the server-side, it first does a
             * get to find the latest versions timestamp.  Then it adds a delete using
             * the fetched cells timestamp.
             * @param family family name
             * @param qualifier column qualifier
             * @return this for invocation chaining
             */
            public Delete addColumn(final byte [] family, final byte [] qualifier) {
              this.addColumn(family, qualifier, this.ts);
              return this;
            }
          

          In server-side, it will get the latest version. And it use setMaxVersions and DeleteVersionVisibilityExpressionFilter to get the latest version. But after this fix, setMaxVersions can not make sure to return the specified number of versions. So there is nothing to delete.
          Code in prePrepareTimeStampForDeleteVersion method of org.apache.hadoop.hbase.security.visibility.VisibilityController.

              get.setFilter(new DeleteVersionVisibilityExpressionFilter(visibilityTags,
                  VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT));
              List<Cell> result = ctx.getEnvironment().getRegion().get(get, false);
          
              if (result.size() < get.getMaxVersions()) {
                // Nothing to delete
                CellUtil.updateLatestStamp(cell, byteNow, 0);
                return;
              }
              if (result.size() > get.getMaxVersions()) {
                throw new RuntimeException("Unexpected size: " + result.size()
                    + ". Results more than the max versions obtained.");
              }
              Cell getCell = result.get(get.getMaxVersions() - 1);
              CellUtil.setTimestamp(cell, getCell.getTimestamp());
          
          Show
          zghaobac Guanghao Zhang added a comment - Failed ut TestVisibilityLabelsWithDeletes is related. The reason is about the Delete the latest version of the specified column. /** * Delete the latest version of the specified column. * This is an expensive call in that on the server-side, it first does a * get to find the latest versions timestamp. Then it adds a delete using * the fetched cells timestamp. * @param family family name * @param qualifier column qualifier * @ return this for invocation chaining */ public Delete addColumn( final byte [] family, final byte [] qualifier) { this .addColumn(family, qualifier, this .ts); return this ; } In server-side, it will get the latest version. And it use setMaxVersions and DeleteVersionVisibilityExpressionFilter to get the latest version. But after this fix, setMaxVersions can not make sure to return the specified number of versions. So there is nothing to delete. Code in prePrepareTimeStampForDeleteVersion method of org.apache.hadoop.hbase.security.visibility.VisibilityController. get.setFilter( new DeleteVersionVisibilityExpressionFilter(visibilityTags, VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT)); List<Cell> result = ctx.getEnvironment().getRegion().get(get, false ); if (result.size() < get.getMaxVersions()) { // Nothing to delete CellUtil.updateLatestStamp(cell, byteNow, 0); return ; } if (result.size() > get.getMaxVersions()) { throw new RuntimeException( "Unexpected size: " + result.size() + ". Results more than the max versions obtained." ); } Cell getCell = result.get(get.getMaxVersions() - 1); CellUtil.setTimestamp(cell, getCell.getTimestamp());
          Hide
          zghaobac Guanghao Zhang added a comment -

          Upload a 003 patch to fix the ut. The new patch will set max versions to Integer.MAX to get latest version and use count to get the latest timestamp. So add a param 'count' to VisibilityController#prePrepareTimeStampForDeleteVersion.

              // This get's max version is Integer.MAX
              get.setFilter(new DeleteVersionVisibilityExpressionFilter(visibilityTags,
                  VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT));
              List<Cell> result = ctx.getEnvironment().getRegion().get(get, false);
          
              if (result.size() < count) {
                // Nothing to delete
                CellUtil.updateLatestStamp(cell, byteNow, 0);
                return;
              }
              // Use 'count' to get the latest cell and update the timestamp of delete
              Cell getCell = result.get(count - 1);
              CellUtil.setTimestamp(cell, getCell.getTimestamp());
          

          Meanwhile, the patch update the javadoc of setMaxVersions in Scan and Get. And I add 2 ut about delete specified version of the specified column and delete the latest version of the specified column.

          Show
          zghaobac Guanghao Zhang added a comment - Upload a 003 patch to fix the ut. The new patch will set max versions to Integer.MAX to get latest version and use count to get the latest timestamp. So add a param 'count' to VisibilityController#prePrepareTimeStampForDeleteVersion. // This get's max version is Integer .MAX get.setFilter( new DeleteVersionVisibilityExpressionFilter(visibilityTags, VisibilityConstants.SORTED_ORDINAL_SERIALIZATION_FORMAT)); List<Cell> result = ctx.getEnvironment().getRegion().get(get, false ); if (result.size() < count) { // Nothing to delete CellUtil.updateLatestStamp(cell, byteNow, 0); return ; } // Use 'count' to get the latest cell and update the timestamp of delete Cell getCell = result.get(count - 1); CellUtil.setTimestamp(cell, getCell.getTimestamp()); Meanwhile, the patch update the javadoc of setMaxVersions in Scan and Get. And I add 2 ut about delete specified version of the specified column and delete the latest version of the specified column.
          Hide
          zghaobac Guanghao Zhang added a comment -

          ramkrishna.s.vasudevan Can you help to review the change about "Deletes with Cell Visibility"? Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - ramkrishna.s.vasudevan Can you help to review the change about "Deletes with Cell Visibility"? Thanks.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 18s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 4 new or modified test files.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 3m 10s master passed
          +1 compile 0m 56s master passed
          +1 checkstyle 0m 47s master passed
          +1 mvneclipse 0m 25s master passed
          +1 findbugs 2m 41s master passed
          +1 javadoc 0m 46s master passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 1m 3s the patch passed
          +1 compile 0m 57s the patch passed
          +1 javac 0m 57s the patch passed
          +1 checkstyle 0m 47s the patch passed
          +1 mvneclipse 0m 25s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 29m 24s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 3m 3s the patch passed
          +1 javadoc 0m 46s the patch passed
          +1 unit 2m 19s hbase-client in the patch passed.
          -1 unit 100m 1s hbase-server in the patch failed.
          +1 asflicense 0m 28s The patch does not generate ASF License warnings.
          149m 13s



          Reason Tests
          Failed junit tests hadoop.hbase.coprocessor.TestRegionObserverInterface



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858654/HBASE-17125.master.003.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux b72559fae53a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 44b2558
          Default Java 1.8.0_121
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6093/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6093/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 18s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 4 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 3m 10s master passed +1 compile 0m 56s master passed +1 checkstyle 0m 47s master passed +1 mvneclipse 0m 25s master passed +1 findbugs 2m 41s master passed +1 javadoc 0m 46s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 1m 3s the patch passed +1 compile 0m 57s the patch passed +1 javac 0m 57s the patch passed +1 checkstyle 0m 47s the patch passed +1 mvneclipse 0m 25s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 29m 24s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 3m 3s the patch passed +1 javadoc 0m 46s the patch passed +1 unit 2m 19s hbase-client in the patch passed. -1 unit 100m 1s hbase-server in the patch failed. +1 asflicense 0m 28s The patch does not generate ASF License warnings. 149m 13s Reason Tests Failed junit tests hadoop.hbase.coprocessor.TestRegionObserverInterface Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858654/HBASE-17125.master.003.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux b72559fae53a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 44b2558 Default Java 1.8.0_121 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6093/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6093/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6093/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Update javadoc of setMaxVersions in 005 patch.

          Show
          zghaobac Guanghao Zhang added a comment - Update javadoc of setMaxVersions in 005 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 28s Maven dependency ordering for branch
          +1 mvninstall 3m 14s master passed
          +1 compile 0m 53s master passed
          +1 checkstyle 0m 44s master passed
          +1 mvneclipse 0m 23s master passed
          +1 findbugs 2m 32s master passed
          +1 javadoc 0m 43s master passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 44s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 28m 3s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 3m 23s the patch passed
          +1 javadoc 0m 51s the patch passed
          +1 unit 2m 28s hbase-client in the patch passed.
          +1 unit 102m 31s hbase-server in the patch passed.
          +1 asflicense 0m 32s The patch does not generate ASF License warnings.
          150m 44s



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858821/HBASE-17125.master.004.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 1bc81a5f7bd0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / aace02a
          Default Java 1.8.0_121
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6101/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6101/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 28s Maven dependency ordering for branch +1 mvninstall 3m 14s master passed +1 compile 0m 53s master passed +1 checkstyle 0m 44s master passed +1 mvneclipse 0m 23s master passed +1 findbugs 2m 32s master passed +1 javadoc 0m 43s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 59s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 44s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 28m 3s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 3m 23s the patch passed +1 javadoc 0m 51s the patch passed +1 unit 2m 28s hbase-client in the patch passed. +1 unit 102m 31s hbase-server in the patch passed. +1 asflicense 0m 32s The patch does not generate ASF License warnings. 150m 44s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858821/HBASE-17125.master.004.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 1bc81a5f7bd0 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / aace02a Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6101/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6101/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 17s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 22s Maven dependency ordering for branch
          +1 mvninstall 3m 1s master passed
          +1 compile 0m 51s master passed
          +1 checkstyle 0m 45s master passed
          +1 mvneclipse 0m 23s master passed
          +1 findbugs 2m 29s master passed
          +1 javadoc 0m 42s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 0m 59s the patch passed
          +1 compile 0m 51s the patch passed
          +1 javac 0m 51s the patch passed
          +1 checkstyle 0m 43s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 26m 51s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 45s the patch passed
          +1 javadoc 0m 42s the patch passed
          +1 unit 2m 17s hbase-client in the patch passed.
          +1 unit 94m 43s hbase-server in the patch passed.
          +1 asflicense 0m 33s The patch does not generate ASF License warnings.
          140m 22s



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858829/HBASE-17125.master.005.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux c7876db4dcea 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / aace02a
          Default Java 1.8.0_121
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6103/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6103/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 17s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 22s Maven dependency ordering for branch +1 mvninstall 3m 1s master passed +1 compile 0m 51s master passed +1 checkstyle 0m 45s master passed +1 mvneclipse 0m 23s master passed +1 findbugs 2m 29s master passed +1 javadoc 0m 42s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 0m 59s the patch passed +1 compile 0m 51s the patch passed +1 javac 0m 51s the patch passed +1 checkstyle 0m 43s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 26m 51s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 45s the patch passed +1 javadoc 0m 42s the patch passed +1 unit 2m 17s hbase-client in the patch passed. +1 unit 94m 43s hbase-server in the patch passed. +1 asflicense 0m 33s The patch does not generate ASF License warnings. 140m 22s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12858829/HBASE-17125.master.005.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux c7876db4dcea 3.13.0-107-generic #154-Ubuntu SMP Tue Dec 20 09:57:27 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / aace02a Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6103/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6103/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          Will take a look at the change in visibility specifcally. Thanks.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - Will take a look at the change in visibility specifcally. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Ping Duo Zhang...

          Show
          zghaobac Guanghao Zhang added a comment - Ping Duo Zhang ...
          Hide
          zghaobac Guanghao Zhang added a comment -

          Open a issue HBASE-17797 to introduce a filter to implement read the special number of versions of column.

          Show
          zghaobac Guanghao Zhang added a comment - Open a issue HBASE-17797 to introduce a filter to implement read the special number of versions of column.
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 10s Maven dependency ordering for branch
          +1 mvninstall 3m 0s master passed
          +1 compile 0m 51s master passed
          +1 checkstyle 0m 43s master passed
          +1 mvneclipse 0m 23s master passed
          +1 findbugs 2m 30s master passed
          +1 javadoc 0m 41s master passed
          0 mvndep 0m 11s Maven dependency ordering for patch
          +1 mvninstall 0m 58s the patch passed
          +1 compile 0m 53s the patch passed
          +1 javac 0m 53s the patch passed
          +1 checkstyle 0m 44s the patch passed
          +1 mvneclipse 0m 23s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 hadoopcheck 26m 59s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 findbugs 2m 45s the patch passed
          +1 javadoc 0m 42s the patch passed
          +1 unit 2m 14s hbase-client in the patch passed.
          +1 unit 99m 33s hbase-server in the patch passed.
          +1 asflicense 0m 30s The patch does not generate ASF License warnings.
          144m 59s



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859270/HBASE-17125.master.006.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile
          uname Linux 7392b7ea497a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 0ed50f4
          Default Java 1.8.0_121
          findbugs v3.0.0
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6130/testReport/
          modules C: hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6130/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 10s Maven dependency ordering for branch +1 mvninstall 3m 0s master passed +1 compile 0m 51s master passed +1 checkstyle 0m 43s master passed +1 mvneclipse 0m 23s master passed +1 findbugs 2m 30s master passed +1 javadoc 0m 41s master passed 0 mvndep 0m 11s Maven dependency ordering for patch +1 mvninstall 0m 58s the patch passed +1 compile 0m 53s the patch passed +1 javac 0m 53s the patch passed +1 checkstyle 0m 44s the patch passed +1 mvneclipse 0m 23s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 hadoopcheck 26m 59s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 findbugs 2m 45s the patch passed +1 javadoc 0m 42s the patch passed +1 unit 2m 14s hbase-client in the patch passed. +1 unit 99m 33s hbase-server in the patch passed. +1 asflicense 0m 30s The patch does not generate ASF License warnings. 144m 59s Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12859270/HBASE-17125.master.006.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile uname Linux 7392b7ea497a 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 0ed50f4 Default Java 1.8.0_121 findbugs v3.0.0 Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6130/testReport/ modules C: hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6130/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Stack Lars Hofhansl ramkrishna.s.vasudevan Can you help review? Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - Stack Lars Hofhansl ramkrishna.s.vasudevan Can you help review? Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -
          Show
          zghaobac Guanghao Zhang added a comment - ping ramkrishna.s.vasudevan
          Hide
          ram_krish ramkrishna.s.vasudevan added a comment -

          W.r.t to the change in the visibility controller it seems fine because instead of getting the maxVersions that was set in the Get/scan you are passing it now as the count param via the CPs..

          Set the maximum versions of each column will be check by filter. When use filter, the get

          257 * maybe return less than maximum versions for each column.

          Better to rephrase this like
          'Sets the maximum versions of each column that will be checked by the filters. When filters are in use, the get may return less than maximum versions for each column'.

          The rest of the patch need to see.

          Show
          ram_krish ramkrishna.s.vasudevan added a comment - W.r.t to the change in the visibility controller it seems fine because instead of getting the maxVersions that was set in the Get/scan you are passing it now as the count param via the CPs.. Set the maximum versions of each column will be check by filter. When use filter, the get 257 * maybe return less than maximum versions for each column. Better to rephrase this like 'Sets the maximum versions of each column that will be checked by the filters. When filters are in use, the get may return less than maximum versions for each column'. The rest of the patch need to see.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Joining here late.
          So this is a behave change right? To check for the versions first and then apply filter on top? There might be usages depending on the old behave?
          Just a use case. Am using visibility labels and having many values for the same key (Many cells with diff labels). A user having specific visibility label auth doing a get and it is a simple Get() request means want to see only one latest version for which I have label permission. Now after this patch, the versions is applied which will just give one version and that might not have a label corresponding to this user. So he will see no results! Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters. What to do here?

          Show
          anoop.hbase Anoop Sam John added a comment - Joining here late. So this is a behave change right? To check for the versions first and then apply filter on top? There might be usages depending on the old behave? Just a use case. Am using visibility labels and having many values for the same key (Many cells with diff labels). A user having specific visibility label auth doing a get and it is a simple Get() request means want to see only one latest version for which I have label permission. Now after this patch, the versions is applied which will just give one version and that might not have a label corresponding to this user. So he will see no results! Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters. What to do here?
          Hide
          zghaobac Guanghao Zhang added a comment -

          So this is a behave change right? To check for the versions first and then apply filter on top?

          Yeah, this patch will change setMaxVersions()'s behavior.

          Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters.

          I thought this can be fixed by HBASE-17797. When use auth and ACL, we can add a filter which return special number of versions for Get/Scan.

          Show
          zghaobac Guanghao Zhang added a comment - So this is a behave change right? To check for the versions first and then apply filter on top? Yeah, this patch will change setMaxVersions()'s behavior. Previously we have seeing the cell for which he had label auth. Same way per cell ACL also have similar issue. Both of these features implemented via Filters. I thought this can be fixed by HBASE-17797 . When use auth and ACL, we can add a filter which return special number of versions for Get/Scan.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Some crazy thinking.
          May be we should check the versions thing in 2 steps. We have max versions to keep per column in table definition. And every read request will have #versions that it would like to see.
          We should do the version number filter for the table def level versions# at first.
          Then Filter ops (As done today)
          Finally a check on the #versions as specified by the read op and return only specific #cells.
          Just wild thought. Dont know the feasibility and will add much more overhead or not.

          Show
          anoop.hbase Anoop Sam John added a comment - Some crazy thinking. May be we should check the versions thing in 2 steps. We have max versions to keep per column in table definition. And every read request will have #versions that it would like to see. We should do the version number filter for the table def level versions# at first. Then Filter ops (As done today) Finally a check on the #versions as specified by the read op and return only specific #cells. Just wild thought. Dont know the feasibility and will add much more overhead or not.
          Hide
          Apache9 Duo Zhang added a comment -

          Anoop Sam John We have considered this solution before. See this comment

          https://issues.apache.org/jira/browse/HBASE-17125?focusedCommentId=15889451&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15889451

          But I do not think it is necessary. Our documentation never says we will do things like this. It will make the implementation much more complicated. And if user still want this feature, it can be addressed by HBASE-17797.

          Here seems Guanghao Zhang want to address the problem of visivility labels and ACLs by always read all the versions. It will have bad impact on performance. So maybe we could close HBASE-17797 and just implement the max version filter here and give an example for how to use the filter?
          WDYT Guanghao Zhang?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Anoop Sam John We have considered this solution before. See this comment https://issues.apache.org/jira/browse/HBASE-17125?focusedCommentId=15889451&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-15889451 But I do not think it is necessary. Our documentation never says we will do things like this. It will make the implementation much more complicated. And if user still want this feature, it can be addressed by HBASE-17797 . Here seems Guanghao Zhang want to address the problem of visivility labels and ACLs by always read all the versions. It will have bad impact on performance. So maybe we could close HBASE-17797 and just implement the max version filter here and give an example for how to use the filter? WDYT Guanghao Zhang ? Thanks.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          Sorry did not read all the discuss in this jira. I came in late. As said just I was saying. Would like to see the patch for solving the ACL/labels issue.

          Show
          anoop.hbase Anoop Sam John added a comment - Sorry did not read all the discuss in this jira. I came in late. As said just I was saying. Would like to see the patch for solving the ACL/labels issue.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Add 007 patch which implement a new SpecifiedNumVersionsColumnFilter. And the ut testSpecifiedNumVersionsColumnFilter shows how to use this filter.

          Show
          zghaobac Guanghao Zhang added a comment - Add 007 patch which implement a new SpecifiedNumVersionsColumnFilter. And the ut testSpecifiedNumVersionsColumnFilter shows how to use this filter.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 31s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 43s Maven dependency ordering for branch
          +1 mvninstall 7m 39s master passed
          +1 compile 3m 32s master passed
          +1 checkstyle 20m 31s master passed
          +1 mvneclipse 0m 50s master passed
          -1 findbugs 4m 36s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 1m 32s master passed
          0 mvndep 0m 19s Maven dependency ordering for patch
          +1 mvninstall 3m 1s the patch passed
          +1 compile 2m 47s the patch passed
          +1 cc 2m 47s the patch passed
          +1 javac 2m 47s the patch passed
          +1 checkstyle 20m 35s the patch passed
          +1 mvneclipse 1m 0s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 61m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 2m 29s the patch passed
          +1 findbugs 13m 8s the patch passed
          +1 javadoc 1m 32s the patch passed
          +1 unit 0m 59s hbase-protocol-shaded in the patch passed.
          -1 unit 1m 47s hbase-client in the patch failed.
          -1 unit 168m 0s hbase-server in the patch failed.
          +1 asflicense 1m 13s The patch does not generate ASF License warnings.
          325m 22s



          Reason Tests
          Failed junit tests hadoop.hbase.TestInterfaceAudienceAnnotations
            hadoop.hbase.snapshot.TestExportSnapshot
            hadoop.hbase.client.TestAsyncBalancerAdminApi
            hadoop.hbase.snapshot.TestSecureExportSnapshot
            hadoop.hbase.snapshot.TestMobExportSnapshot
            hadoop.hbase.snapshot.TestMobSecureExportSnapshot
          Timed out junit tests org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence
            org.apache.hadoop.hbase.master.TestMetaShutdownHandler



          Subsystem Report/Notes
          Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863201/HBASE-17125.master.007.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 6a6d517f01fc 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / da5fb27
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6410/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6410/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 31s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 43s Maven dependency ordering for branch +1 mvninstall 7m 39s master passed +1 compile 3m 32s master passed +1 checkstyle 20m 31s master passed +1 mvneclipse 0m 50s master passed -1 findbugs 4m 36s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 1m 32s master passed 0 mvndep 0m 19s Maven dependency ordering for patch +1 mvninstall 3m 1s the patch passed +1 compile 2m 47s the patch passed +1 cc 2m 47s the patch passed +1 javac 2m 47s the patch passed +1 checkstyle 20m 35s the patch passed +1 mvneclipse 1m 0s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 61m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 2m 29s the patch passed +1 findbugs 13m 8s the patch passed +1 javadoc 1m 32s the patch passed +1 unit 0m 59s hbase-protocol-shaded in the patch passed. -1 unit 1m 47s hbase-client in the patch failed. -1 unit 168m 0s hbase-server in the patch failed. +1 asflicense 1m 13s The patch does not generate ASF License warnings. 325m 22s Reason Tests Failed junit tests hadoop.hbase.TestInterfaceAudienceAnnotations   hadoop.hbase.snapshot.TestExportSnapshot   hadoop.hbase.client.TestAsyncBalancerAdminApi   hadoop.hbase.snapshot.TestSecureExportSnapshot   hadoop.hbase.snapshot.TestMobExportSnapshot   hadoop.hbase.snapshot.TestMobSecureExportSnapshot Timed out junit tests org.apache.hadoop.hbase.master.TestMasterFailoverBalancerPersistence   org.apache.hadoop.hbase.master.TestMetaShutdownHandler Subsystem Report/Notes Docker Client=17.03.0-ce Server=17.03.0-ce Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863201/HBASE-17125.master.007.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 6a6d517f01fc 4.8.3-std-1 #1 SMP Fri Oct 21 11:15:43 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / da5fb27 Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-client.txt https://builds.apache.org/job/PreCommit-HBASE-Build/6410/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6410/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6410/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Add InterfaceAudience for SpecifiedNumVersionsColumnFilter. The failed ut passed locally. TestExportSnapshot* failed in other pre-commit jobs, too. So open a issue https://issues.apache.org/jira/browse/HBASE-17913 for it.

          Show
          zghaobac Guanghao Zhang added a comment - Add InterfaceAudience for SpecifiedNumVersionsColumnFilter. The failed ut passed locally. TestExportSnapshot* failed in other pre-commit jobs, too. So open a issue https://issues.apache.org/jira/browse/HBASE-17913 for it.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 30s Maven dependency ordering for branch
          +1 mvninstall 3m 17s master passed
          +1 compile 1m 31s master passed
          +1 checkstyle 10m 58s master passed
          +1 mvneclipse 0m 35s master passed
          -1 findbugs 2m 9s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 1m 0s master passed
          0 mvndep 0m 13s Maven dependency ordering for patch
          +1 mvninstall 1m 45s the patch passed
          +1 compile 1m 30s the patch passed
          +1 cc 1m 30s the patch passed
          +1 javac 1m 30s the patch passed
          +1 checkstyle 11m 16s the patch passed
          +1 mvneclipse 0m 32s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 29m 35s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 1m 5s the patch passed
          +1 findbugs 5m 50s the patch passed
          +1 javadoc 0m 54s the patch passed
          +1 unit 0m 29s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 18s hbase-client in the patch passed.
          -1 unit 109m 14s hbase-server in the patch failed.
          +1 asflicense 0m 48s The patch does not generate ASF License warnings.
          189m 49s



          Reason Tests
          Failed junit tests hadoop.hbase.snapshot.TestMobSecureExportSnapshot
            hadoop.hbase.snapshot.TestMobExportSnapshot
            hadoop.hbase.snapshot.TestSecureExportSnapshot



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863413/HBASE-17125.master.008.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 0cbf4c853473 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / da5fb27
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6420/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6420/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 30s Maven dependency ordering for branch +1 mvninstall 3m 17s master passed +1 compile 1m 31s master passed +1 checkstyle 10m 58s master passed +1 mvneclipse 0m 35s master passed -1 findbugs 2m 9s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 1m 0s master passed 0 mvndep 0m 13s Maven dependency ordering for patch +1 mvninstall 1m 45s the patch passed +1 compile 1m 30s the patch passed +1 cc 1m 30s the patch passed +1 javac 1m 30s the patch passed +1 checkstyle 11m 16s the patch passed +1 mvneclipse 0m 32s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 29m 35s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 1m 5s the patch passed +1 findbugs 5m 50s the patch passed +1 javadoc 0m 54s the patch passed +1 unit 0m 29s hbase-protocol-shaded in the patch passed. +1 unit 2m 18s hbase-client in the patch passed. -1 unit 109m 14s hbase-server in the patch failed. +1 asflicense 0m 48s The patch does not generate ASF License warnings. 189m 49s Reason Tests Failed junit tests hadoop.hbase.snapshot.TestMobSecureExportSnapshot   hadoop.hbase.snapshot.TestMobExportSnapshot   hadoop.hbase.snapshot.TestSecureExportSnapshot Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12863413/HBASE-17125.master.008.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 0cbf4c853473 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / da5fb27 Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6420/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6420/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6420/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Anoop Sam John Sorry to upload patch late. Please help to review the 008 patch. Review board: https://reviews.apache.org/r/57449/ . Thanks.

          Show
          zghaobac Guanghao Zhang added a comment - Anoop Sam John Sorry to upload patch late. Please help to review the 008 patch. Review board: https://reviews.apache.org/r/57449/ . Thanks.
          Hide
          Apache9 Duo Zhang added a comment -

          Ping Anoop Sam John. I'm +1 on the latest patch.

          Show
          Apache9 Duo Zhang added a comment - Ping Anoop Sam John . I'm +1 on the latest patch.
          Hide
          yuzhihong@gmail.com Ted Yu added a comment - - edited

          has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore.

          How is the above addressed in the current patch ?

          Currently the max versions is obtained this way (see UserScanQueryMatcher):

              int maxVersions = scan.isRaw() ? scan.getMaxVersions()
                  : Math.min(scan.getMaxVersions(), scanInfo.getMaxVersions());
          

          The column tracker loses some information when column's max versions is greater than that specified in the Scan.
          Can we pass this information (let's call it the slack) to ColumnTracker ctor ?

          When filterResponse is SKIP, we can utilize the extra information to address the scenario described above by calling a new method of ColumnTracker (let's call it retract which decrements currentCount field if slack permits).

          Show
          yuzhihong@gmail.com Ted Yu added a comment - - edited has another problem, if a column's max version is 5 and the user query only need 3 versions. It first check the version's number, then check the cell by filter. So the cells number of the result may less than 3. But there are 2 versions which don't read anymore. How is the above addressed in the current patch ? Currently the max versions is obtained this way (see UserScanQueryMatcher): int maxVersions = scan.isRaw() ? scan.getMaxVersions() : Math .min(scan.getMaxVersions(), scanInfo.getMaxVersions()); The column tracker loses some information when column's max versions is greater than that specified in the Scan. Can we pass this information (let's call it the slack) to ColumnTracker ctor ? When filterResponse is SKIP, we can utilize the extra information to address the scenario described above by calling a new method of ColumnTracker (let's call it retract which decrements currentCount field if slack permits).
          Hide
          zghaobac Guanghao Zhang added a comment -

          How is the above addressed in the current patch ?

          Now scan.setMaxVerrsions means how many versions will be check. So this can be addressed by scan.setMaxVersions(5).

          Show
          zghaobac Guanghao Zhang added a comment - How is the above addressed in the current patch ? Now scan.setMaxVerrsions means how many versions will be check. So this can be addressed by scan.setMaxVersions(5).
          Hide
          zghaobac Guanghao Zhang added a comment -

          Can we pass this information (let's call it the slack) to ColumnTracker ctor ?

          The javadoc of scan.setMaxVersions has been changed. If user set max versions is less than the column's max versions, it means user didn't want to check all versions. So I thought we don't need this information?

          Show
          zghaobac Guanghao Zhang added a comment - Can we pass this information (let's call it the slack) to ColumnTracker ctor ? The javadoc of scan.setMaxVersions has been changed. If user set max versions is less than the column's max versions, it means user didn't want to check all versions. So I thought we don't need this information?
          Hide
          yuzhihong@gmail.com Ted Yu added a comment -

          Let's look at the scenario again:

          if a column's max version is 5 and the user query only need 3 versions

          If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned.
          Instead of letting user deal with the slack, it would be better to handle this server side.

          My proposal only involves a few lines of change to your latest patch - though there may be some unit test failure(s).

          Show
          yuzhihong@gmail.com Ted Yu added a comment - Let's look at the scenario again: if a column's max version is 5 and the user query only need 3 versions If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned. Instead of letting user deal with the slack, it would be better to handle this server side. My proposal only involves a few lines of change to your latest patch - though there may be some unit test failure(s).
          Hide
          zghaobac Guanghao Zhang added a comment -

          If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned.

          This can be addressed by scan.setFilter(new SpecifiedNumVersionsColumnFilter(3)). setMaxVersions means how many version will be check. And SpecifiedNumVersionsColumnFilter means how many versions will be returned.

          Show
          zghaobac Guanghao Zhang added a comment - If user calls scan#setMaxVersions(5), server would check more versions (than 3). However, there is a chance that more than 3 versions would be returned. This can be addressed by scan.setFilter(new SpecifiedNumVersionsColumnFilter(3)). setMaxVersions means how many version will be check. And SpecifiedNumVersionsColumnFilter means how many versions will be returned.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          I dont think it is correct to ask users to do setMaxVersions(5). It will so complicated for a user to set this as 5 and then the filter with 3. This is like we pass our impl headache to user. IMHO.

          Show
          anoop.hbase Anoop Sam John added a comment - I dont think it is correct to ask users to do setMaxVersions(5). It will so complicated for a user to set this as 5 and then the filter with 3. This is like we pass our impl headache to user. IMHO.
          Hide
          Apache9 Duo Zhang added a comment -

          Why users need to setMaxVerions to 5 if they only want 3 versions? The fact is, if you do not use filter, then just use setMaxVersions to control the number of the veraions returned. If you use filter, and then please use SpecifiedNumVersionsColumnFilter if you want to control the number of versions returned as the max versions will be tested before filter. The setMaxVersions is used to control the number of versions passed to filter. I think this is clear enough?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Why users need to setMaxVerions to 5 if they only want 3 versions? The fact is, if you do not use filter, then just use setMaxVersions to control the number of the veraions returned. If you use filter, and then please use SpecifiedNumVersionsColumnFilter if you want to control the number of versions returned as the max versions will be tested before filter. The setMaxVersions is used to control the number of versions passed to filter. I think this is clear enough? Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          It will so complicated for a user to set this as 5 and then the filter with 3.

          The default version is 1. So the scan will only check the latest version. The user need set a bigger value if he want read more than one version. This scenario: the column's versions is 5. So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions.

          Show
          zghaobac Guanghao Zhang added a comment - It will so complicated for a user to set this as 5 and then the filter with 3. The default version is 1. So the scan will only check the latest version. The user need set a bigger value if he want read more than one version. This scenario: the column's versions is 5. So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions.
          Hide
          Apache9 Duo Zhang added a comment -

          So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions.

          Yes, just call scan.setMaxVerions(), do not need to give a specific value.

          Show
          Apache9 Duo Zhang added a comment - So setMaxVersions with any value >= 5 (not only 5), then the server can check all versions. Yes, just call scan.setMaxVerions(), do not need to give a specific value.
          Hide
          anoop.hbase Anoop Sam John added a comment -

          In user@ mailing list there was a query from a user regarding similar issue while using value filter. The recent value of a cell is not matching the value as per filter still he gets a older version of the cell.. He needs only one version (latest). So there also the user has to set this filter on get/scan and setMaxversions()?

          Show
          anoop.hbase Anoop Sam John added a comment - In user@ mailing list there was a query from a user regarding similar issue while using value filter. The recent value of a cell is not matching the value as per filter still he gets a older version of the cell.. He needs only one version (latest). So there also the user has to set this filter on get/scan and setMaxversions()?
          Hide
          Apache9 Duo Zhang added a comment -

          So there also the user has to set this filter on get/scan and setMaxversions()?

          No. After the patch here the problem is gone. Just keep the code, everything will be OK.

          And the problem of this fix is that we can not use setMaxVerions to control the number of returned versions if you user filter(maybe). So we introduce a SpecifiedNumVersionsColumnFilter to solve the problem.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - So there also the user has to set this filter on get/scan and setMaxversions()? No. After the patch here the problem is gone. Just keep the code, everything will be OK. And the problem of this fix is that we can not use setMaxVerions to control the number of returned versions if you user filter(maybe). So we introduce a SpecifiedNumVersionsColumnFilter to solve the problem. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          For the user mailing list case, the column's version is 1. So the user didn't need to setMaxVersions(). If the recent value is not matching, he will gets nothing.

          Show
          zghaobac Guanghao Zhang added a comment - For the user mailing list case, the column's version is 1. So the user didn't need to setMaxVersions(). If the recent value is not matching, he will gets nothing.
          Hide
          Apache9 Duo Zhang added a comment -

          Oh, seems the user calls setMaxVerions to 1. I believe the problem is that he/she found that the filter will return old values then he/she use setMaxVersions(1) and hope this could solve the problem.

          So it is clear that in this user's mind, setMaxVersions should be used to control the number of versions passed to the filter. This is exactly what we provide in the latest patch. With the patch in place, the user does not need to call setMaxVersions(1) anymore.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - Oh, seems the user calls setMaxVerions to 1. I believe the problem is that he/she found that the filter will return old values then he/she use setMaxVersions(1) and hope this could solve the problem. So it is clear that in this user's mind, setMaxVersions should be used to control the number of versions passed to the filter. This is exactly what we provide in the latest patch. With the patch in place, the user does not need to call setMaxVersions(1) anymore. Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          The right javadoc of setMaxVersions is:
          /**

          • If not use filter, get up to the specified number of versions of each column.
          • If use filter, it means the maximum versions of each column will be checked by filter. So the scan
          • maybe return less than maximum versions for each column. But you can add a SpecifiedNumVersionsColumnFilter
          • to get the specified number of versions of each column.
          • @param maxVersions maximum versions for each column
          • @return this
            */
          Show
          zghaobac Guanghao Zhang added a comment - The right javadoc of setMaxVersions is: /** If not use filter, get up to the specified number of versions of each column. If use filter, it means the maximum versions of each column will be checked by filter. So the scan maybe return less than maximum versions for each column. But you can add a SpecifiedNumVersionsColumnFilter to get the specified number of versions of each column. @param maxVersions maximum versions for each column @return this */
          Hide
          zghaobac Guanghao Zhang added a comment -

          Update the javadoc for setMaxVersions in 009 patch.

          Show
          zghaobac Guanghao Zhang added a comment - Update the javadoc for setMaxVersions in 009 patch.
          Hide
          Apache9 Duo Zhang added a comment -

          If not use filter, get up to the specified number of versions of each column.
          <p>
          If use filter, it means the maximum versions of each column will be checked by filter. So the scan may return less than the value you set here as the filter may filter out some cells. If you want to get a specific number of version for each column after filtering, please call setMaxVersions() and use SpecifiedNumVersionsColumnFilter. Notice that the SpecifiedNumVersionsColumnFilter should be placed at the last position in FilterList to make sure it will be checked at last.

          Better? And seems get also has this method so you need to change the wording from scan to get if possible?

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - If not use filter, get up to the specified number of versions of each column. <p> If use filter, it means the maximum versions of each column will be checked by filter. So the scan may return less than the value you set here as the filter may filter out some cells. If you want to get a specific number of version for each column after filtering, please call setMaxVersions() and use SpecifiedNumVersionsColumnFilter. Notice that the SpecifiedNumVersionsColumnFilter should be placed at the last position in FilterList to make sure it will be checked at last. Better? And seems get also has this method so you need to change the wording from scan to get if possible? Thanks.
          Hide
          zghaobac Guanghao Zhang added a comment -

          Update the javadoc in the latest 009 patch.

          Show
          zghaobac Guanghao Zhang added a comment - Update the javadoc in the latest 009 patch.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 20s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 29s Maven dependency ordering for branch
          +1 mvninstall 3m 26s master passed
          +1 compile 1m 24s master passed
          +1 checkstyle 11m 2s master passed
          +1 mvneclipse 0m 32s master passed
          -1 findbugs 2m 5s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 0m 53s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 33s the patch passed
          +1 compile 1m 29s the patch passed
          +1 cc 1m 29s the patch passed
          +1 javac 1m 29s the patch passed
          +1 checkstyle 11m 2s the patch passed
          +1 mvneclipse 0m 31s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 28m 38s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 1m 2s the patch passed
          +1 findbugs 5m 33s the patch passed
          +1 javadoc 0m 50s the patch passed
          +1 unit 0m 26s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 16s hbase-client in the patch passed.
          -1 unit 112m 22s hbase-server in the patch failed.
          +1 asflicense 0m 48s The patch does not generate ASF License warnings.
          190m 27s



          Reason Tests
          Failed junit tests hadoop.hbase.regionserver.TestWalAndCompactingMemStoreFlush
            hadoop.hbase.snapshot.TestExportSnapshot



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864458/HBASE-17125.master.009.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 2f81ad5ac9e7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / 33dadc1
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6526/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6526/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 20s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 29s Maven dependency ordering for branch +1 mvninstall 3m 26s master passed +1 compile 1m 24s master passed +1 checkstyle 11m 2s master passed +1 mvneclipse 0m 32s master passed -1 findbugs 2m 5s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 0m 53s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 33s the patch passed +1 compile 1m 29s the patch passed +1 cc 1m 29s the patch passed +1 javac 1m 29s the patch passed +1 checkstyle 11m 2s the patch passed +1 mvneclipse 0m 31s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 28m 38s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 1m 2s the patch passed +1 findbugs 5m 33s the patch passed +1 javadoc 0m 50s the patch passed +1 unit 0m 26s hbase-protocol-shaded in the patch passed. +1 unit 2m 16s hbase-client in the patch passed. -1 unit 112m 22s hbase-server in the patch failed. +1 asflicense 0m 48s The patch does not generate ASF License warnings. 190m 27s Reason Tests Failed junit tests hadoop.hbase.regionserver.TestWalAndCompactingMemStoreFlush   hadoop.hbase.snapshot.TestExportSnapshot Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864458/HBASE-17125.master.009.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 2f81ad5ac9e7 3.13.0-106-generic #153-Ubuntu SMP Tue Dec 6 15:44:32 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / 33dadc1 Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6526/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6526/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6526/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          chia7712 Chia-Ping Tsai added a comment -

          TestWalAndCompactingMemStoreFlush is traced by HBASE-17943.

          Show
          chia7712 Chia-Ping Tsai added a comment - TestWalAndCompactingMemStoreFlush is traced by HBASE-17943 .
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 37s Docker mode activated.
          +1 hbaseanti 0m 0s Patch does not have any anti-patterns.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 5 new or modified test files.
          0 mvndep 0m 27s Maven dependency ordering for branch
          +1 mvninstall 3m 37s master passed
          +1 compile 1m 33s master passed
          +1 checkstyle 11m 42s master passed
          +1 mvneclipse 0m 34s master passed
          -1 findbugs 2m 15s hbase-protocol-shaded in master has 24 extant Findbugs warnings.
          +1 javadoc 0m 56s master passed
          0 mvndep 0m 12s Maven dependency ordering for patch
          +1 mvninstall 1m 40s the patch passed
          +1 compile 1m 32s the patch passed
          +1 cc 1m 32s the patch passed
          +1 javac 1m 32s the patch passed
          +1 checkstyle 11m 43s the patch passed
          +1 mvneclipse 0m 34s the patch passed
          -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix.
          +1 hadoopcheck 33m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2.
          +1 hbaseprotoc 1m 18s the patch passed
          +1 findbugs 6m 12s the patch passed
          +1 javadoc 0m 57s the patch passed
          +1 unit 0m 31s hbase-protocol-shaded in the patch passed.
          +1 unit 2m 28s hbase-client in the patch passed.
          -1 unit 121m 53s hbase-server in the patch failed.
          +1 asflicense 0m 51s The patch does not generate ASF License warnings.
          208m 56s



          Reason Tests
          Failed junit tests hadoop.hbase.snapshot.TestMobExportSnapshot
            hadoop.hbase.snapshot.TestExportSnapshot
            hadoop.hbase.snapshot.TestMobSecureExportSnapshot



          Subsystem Report/Notes
          Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864479/HBASE-17125.master.009.patch
          JIRA Issue HBASE-17125
          Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc
          uname Linux 07fc238c0667 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh
          git revision master / d39f40e
          Default Java 1.8.0_121
          findbugs v3.0.0
          findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html
          whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/whitespace-eol.txt
          unit https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt
          unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt
          Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6528/testReport/
          modules C: hbase-protocol-shaded hbase-client hbase-server U: .
          Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6528/console
          Powered by Apache Yetus 0.3.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 37s Docker mode activated. +1 hbaseanti 0m 0s Patch does not have any anti-patterns. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 5 new or modified test files. 0 mvndep 0m 27s Maven dependency ordering for branch +1 mvninstall 3m 37s master passed +1 compile 1m 33s master passed +1 checkstyle 11m 42s master passed +1 mvneclipse 0m 34s master passed -1 findbugs 2m 15s hbase-protocol-shaded in master has 24 extant Findbugs warnings. +1 javadoc 0m 56s master passed 0 mvndep 0m 12s Maven dependency ordering for patch +1 mvninstall 1m 40s the patch passed +1 compile 1m 32s the patch passed +1 cc 1m 32s the patch passed +1 javac 1m 32s the patch passed +1 checkstyle 11m 43s the patch passed +1 mvneclipse 0m 34s the patch passed -1 whitespace 0m 0s The patch has 1 line(s) that end in whitespace. Use git apply --whitespace=fix. +1 hadoopcheck 33m 29s Patch does not cause any errors with Hadoop 2.6.1 2.6.2 2.6.3 2.6.4 2.6.5 2.7.1 2.7.2 2.7.3 or 3.0.0-alpha2. +1 hbaseprotoc 1m 18s the patch passed +1 findbugs 6m 12s the patch passed +1 javadoc 0m 57s the patch passed +1 unit 0m 31s hbase-protocol-shaded in the patch passed. +1 unit 2m 28s hbase-client in the patch passed. -1 unit 121m 53s hbase-server in the patch failed. +1 asflicense 0m 51s The patch does not generate ASF License warnings. 208m 56s Reason Tests Failed junit tests hadoop.hbase.snapshot.TestMobExportSnapshot   hadoop.hbase.snapshot.TestExportSnapshot   hadoop.hbase.snapshot.TestMobSecureExportSnapshot Subsystem Report/Notes Docker Client=1.12.3 Server=1.12.3 Image:yetus/hbase:8d52d23 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12864479/HBASE-17125.master.009.patch JIRA Issue HBASE-17125 Optional Tests asflicense javac javadoc unit findbugs hadoopcheck hbaseanti checkstyle compile cc hbaseprotoc uname Linux 07fc238c0667 3.13.0-105-generic #152-Ubuntu SMP Fri Dec 2 15:37:11 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /home/jenkins/jenkins-slave/workspace/PreCommit-HBASE-Build/component/dev-support/hbase-personality.sh git revision master / d39f40e Default Java 1.8.0_121 findbugs v3.0.0 findbugs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/branch-findbugs-hbase-protocol-shaded-warnings.html whitespace https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/whitespace-eol.txt unit https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt unit test logs https://builds.apache.org/job/PreCommit-HBASE-Build/6528/artifact/patchprocess/patch-unit-hbase-server.txt Test Results https://builds.apache.org/job/PreCommit-HBASE-Build/6528/testReport/ modules C: hbase-protocol-shaded hbase-client hbase-server U: . Console output https://builds.apache.org/job/PreCommit-HBASE-Build/6528/console Powered by Apache Yetus 0.3.0 http://yetus.apache.org This message was automatically generated.
          Hide
          Apache9 Duo Zhang added a comment -

          'If not user filter,' -> 'If not use filter'.

          Any other concerns? Anoop Sam John.

          Thanks.

          Show
          Apache9 Duo Zhang added a comment - 'If not user filter,' -> 'If not use filter'. Any other concerns? Anoop Sam John . Thanks.

            People

            • Assignee:
              zghaobac Guanghao Zhang
              Reporter:
              zghaobac Guanghao Zhang
            • Votes:
              0 Vote for this issue
              Watchers:
              18 Start watching this issue

              Dates

              • Created:
                Updated:

                Development