Details
Description
PhoenixAccessController should use AccessChecker instead of AccessControlClient for permission checks.
In HBase, every RegionServer's AccessController maintains a local cache of permissions. At startup time they are initialized from the ACL table. Whenever the ACL table is changed (via grant or revoke) the AC on the ACL table "broadcasts" the change via zookeeper, which updates the cache. This is performed and managed by TableAuthManager but is exposed as API by AccessChecker. AccessChecker is the result of a refactor that was committed as far back as branch-1.4 I believe.
Phoenix implements its own access controller and is using the client API AccessControlClient instead. AccessControlClient does not cache nor use the ZK-based cache update mechanism, because it is designed for client side use.
The use of AccessControlClient instead of AccessChecker is not scalable. Every permissions check will trigger a remote RPC to the ACL table, which is generally going to be a single region hosted on a single RegionServer.
Attachments
Attachments
- PHOENIX-5269-4.14-HBase-1.4.v2.patch
- 11 kB
- Kiran Kumar Maturi
- PHOENIX-5269-4.14-HBase-1.4.v1.patch
- 11 kB
- Kiran Kumar Maturi
- PHOENIX-5269-4.14-HBase-1.4.patch
- 11 kB
- Kiran Kumar Maturi
- PHOENIX-5269.master.v2.patch
- 19 kB
- Kiran Kumar Maturi
- PHOENIX-5269.master.v1.patch
- 18 kB
- Kiran Kumar Maturi
- PHOENIX-5269.4.x-HBase-1.5.v1.patch
- 17 kB
- Kiran Kumar Maturi
- PHOENIX-5269.4.x-HBase-1.4.v1.patch
- 17 kB
- Kiran Kumar Maturi
- PHOENIX-5269.4.x-HBase-1.3.v1.patch
- 17 kB
- Kiran Kumar Maturi
- PHOENIX-5269.4.14-HBase-1.4.v4.patch
- 17 kB
- Kiran Kumar Maturi
- PHOENIX-5269.4.14-HBase-1.4.v3.patch
- 17 kB
- Kiran Kumar Maturi
- PHOENIX-5269.4.14-HBase-1.3.v1.patch
- 17 kB
- Kiran Kumar Maturi
- diff.patch
- 5 kB
- Thomas D'Silva
Issue Links
- is related to
-
PHOENIX-5267 With namespaces enabled Phoenix client times out with high loads
- Closed
- relates to
-
HBASE-22374 Backport AccessChecker refactor to branch-1.3
- Resolved
-
HBASE-22375 Promote AccessChecker to LimitedPrivate(Coprocessor)
- Resolved
- supercedes
-
PHOENIX-5267 With namespaces enabled Phoenix client times out with high loads
- Closed
-
PHOENIX-5149 Set minimum HBase 1.4 version to 1.4.3
- Resolved
Activity
apurtell
We would want this to be fixed in 1.3, I can file a HBase JIRA if required.
tdsilva apurtell I have made the changes in PhoenixAccessController to use check for permissions using AuthManager provided by AccessChecker. I have tested it running TableDDLPermissionsIT.java it succeeds. I will be doing some more testing and add test cases as well.
tdsilva can you please review.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12968206/PHOENIX-5269-4.14.1-HBase-1.4.patch
against master branch at commit d9057de0e5ec7f1b89b6586f6c03f73163fd01ee.
ATTACHMENT ID: 12968206
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2569//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12968209/PHOENIX-5269-4.14-HBase-1.4.patch
against 4.14-HBase-1.4 branch at commit d9057de0e5ec7f1b89b6586f6c03f73163fd01ee.
ATTACHMENT ID: 12968209
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 javac. The patch appears to cause mvn compile goal to fail .
Compilation errors resume:
[ERROR] COMPILATION ERROR :
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[55,47] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[82,13] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[146,29] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[255,22] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[255,57] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[272,22] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[272,57] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[433,22] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[433,63] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[482,114] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[463,23] cannot find symbol
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[463,64] cannot find symbol
[ERROR] Failed to execute goal org.apache.maven.plugins:maven-compiler-plugin:3.0:compile (default-compile) on project phoenix-core: Compilation failure: Compilation failure:
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[55,47] cannot find symbol
[ERROR] symbol: class AccessChecker
[ERROR] location: package org.apache.hadoop.hbase.security.access
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[82,13] cannot find symbol
[ERROR] symbol: class AccessChecker
[ERROR] location: class org.apache.phoenix.coprocessor.PhoenixAccessController
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[146,29] cannot find symbol
[ERROR] symbol: class AccessChecker
[ERROR] location: class org.apache.phoenix.coprocessor.PhoenixAccessController
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[255,22] cannot find symbol
[ERROR] symbol: class Connection
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[255,57] cannot find symbol
[ERROR] symbol: method createConnection(org.apache.hadoop.conf.Configuration)
[ERROR] location: class org.apache.phoenix.util.ServerUtil.ConnectionFactory
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[272,22] cannot find symbol
[ERROR] symbol: class Connection
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[272,57] cannot find symbol
[ERROR] symbol: method createConnection(org.apache.hadoop.conf.Configuration)
[ERROR] location: class org.apache.phoenix.util.ServerUtil.ConnectionFactory
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[433,22] cannot find symbol
[ERROR] symbol: class Connection
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[433,63] cannot find symbol
[ERROR] symbol: method createConnection(org.apache.hadoop.conf.Configuration)
[ERROR] location: class org.apache.phoenix.util.ServerUtil.ConnectionFactory
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[482,114] cannot find symbol
[ERROR] symbol: class Connection
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[463,23] cannot find symbol
[ERROR] symbol: class Connection
[ERROR] /home/jenkins/jenkins-slave/workspace/PreCommit-PHOENIX-Build/phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java:[463,64] cannot find symbol
[ERROR] symbol: method createConnection(org.apache.hadoop.conf.Configuration)
[ERROR] location: class org.apache.phoenix.util.ServerUtil.ConnectionFactory
[ERROR] -> [Help 1]
[ERROR]
[ERROR] To see the full stack trace of the errors, re-run Maven with the -e switch.
[ERROR] Re-run Maven using the -X switch to enable full debug logging.
[ERROR]
[ERROR] For more information about the errors and possible solutions, please read the following articles:
[ERROR] [Help 1] http://cwiki.apache.org/confluence/display/MAVEN/MojoFailureException
[ERROR]
[ERROR] After correcting the problems, you can resume the build with the command
[ERROR] mvn <goals> -rf :phoenix-core
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2570//console
This message is automatically generated.
It looks like AccessChecker is being called to check if the user or group has access
accessChecker.getAuthManager().userHasAccess
accessChecker.getAuthManager().groupHasAccess
While getting the UserPermissions looks like it is still using AccessControlClient.getUserPermissions, does this patch fix the issue you were seeing in PHOENIX-5267?
tdsilva My assumption is at Phoenix level we can have our own custom access controller. Which will be implement the interface AccessControlClient as i am not sure all implementations will use access checker internally, I am using AccessControlClient.getUserPermissions for those user defined controllers.
With the patch PHOENIX-5267 gets fixed i have tested it.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12968277/PHOENIX-5269-4.14-HBase-1.4.v1.patch
against 4.14-HBase-1.4 branch at commit 700c6436984f23c0a9783e3ea37dd1251b824528.
ATTACHMENT ID: 12968277
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 lineLengths. The patch introduces the following lines longer than 100:
+ if(cp.getClass().getName().equals(org.apache.hadoop.hbase.security.access.AccessController.class.getName())) {
+ List<UserPermission> userPermissions = User.runAsLoginUser(new PrivilegedExceptionAction<List<UserPermission>>() {
+ private void getUserDefinedPermissions(final TableName tableName, final List<UserPermission> userPermissions) throws IOException{
+ try (Connection connection = ConnectionFactory.createConnection(env.getConfiguration())) {
+ if (service.getClass().getName().equals(org.apache.hadoop.hbase.security.access.AccessController.class.getName())) {
+ getUserPermsFromUserDefinedAccessController(userPermissions, connection, (AccessControlService.Interface) service);
+ if(hbaseAccessControllerEnabled && accessChecker.getAuthManager().userHasAccess(user, table, action)) {
+ if(hbaseAccessControllerEnabled && accessChecker.getAuthManager().groupHasAccess(group, table, action)) {
-1 core tests. The patch failed these unit tests:
org.apache.phoenix.hbase.index.covered.TestCoveredColumnIndexCodec
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2571//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2571//console
This message is automatically generated.
kiran.maturi Thanks for working on this.
Is the following code required, looks like userPermissions is not being used anywhere.
List<UserPermission> userPermissions = new ArrayList<>(); if(permissions.length > 0) { getUserDefinedPermissions(tableName, userPermissions); }
With this patch requireAccess will still always call getUserPermissions(tableName) which ends up calling AccessControlClient is that right?
I thought we just wanted to use accessChecker to verify if the user or group has access to a table for the particular action?
I thought we just wanted to use accessChecker to verify if the user or group has access to a table for the particular action?
+1, don't use the AccessControlClient code path as that will not do caching and is not scalable for inline permissions checking while servicing user requests.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12968350/PHOENIX-5269-4.14-HBase-1.4.v2.patch
against 4.14-HBase-1.4 branch at commit 700c6436984f23c0a9783e3ea37dd1251b824528.
ATTACHMENT ID: 12968350
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
-1 lineLengths. The patch introduces the following lines longer than 100:
+ if(cp.getClass().getName().equals(org.apache.hadoop.hbase.security.access.AccessController.class.getName())) {
+ List<UserPermission> userPermissions = User.runAsLoginUser(new PrivilegedExceptionAction<List<UserPermission>>() {
+ private void getUserDefinedPermissions(final TableName tableName, final List<UserPermission> userPermissions) throws IOException{
+ try (Connection connection = ConnectionFactory.createConnection(env.getConfiguration())) {
+ if (service.getClass().getName().equals(org.apache.hadoop.hbase.security.access.AccessController.class.getName())) {
+ getUserPermsFromUserDefinedAccessController(userPermissions, connection, (AccessControlService.Interface) service);
+ if(hbaseAccessControllerEnabled && accessChecker.getAuthManager().userHasAccess(user, table, action)) {
+ if(hbaseAccessControllerEnabled && accessChecker.getAuthManager().groupHasAccess(group, table, action)) {
-1 core tests. The patch failed these unit tests:
org.apache.phoenix.hbase.index.covered.TestCoveredColumnIndexCodec
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2574//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2574//console
This message is automatically generated.
kiran.maturi I know we discussed handling this in a separate JIRA, but if you replace the AccessControlClient calls with AccessChecker do all the tests pass? If so we can just fix it in this patch.
abhishek.chouhan mentioned that you have to close the access checker or else you will be leaking resources.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12968599/PHOENIX-5269.4.14-HBase-1.4.v3.patch
against 4.14-HBase-1.4 branch at commit 746bf1c275461dc5e6622fc004b74044b7ff1b38.
ATTACHMENT ID: 12968599
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2577//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2577//console
This message is automatically generated.
tdsilva AccessControlClient is being used to get/grant permissions and AccessChecker can check/grant for permissions.
Currently for granting permissions to a table/index/view currently we get all (table, group) permissions from hbase:acl and user defined permissions, compute the delta and then grant them using AccessControlClient. We can modify the it to use the current (after patch ) hasAccess Method and check if permissions and grant (using AccessChecker) the required permissions rather than getting all the permissions, that would need more work and will have to consider all the scenarios (Index/View). I think it would be better to do it in a seperate Jira please share your thoughts
abhishek.chouhan thanks for the review. I have uploaded the new patch with the changes.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12968636/PHOENIX-5269.4.14-HBase-1.4.v4.patch
against 4.14-HBase-1.4 branch at commit 746bf1c275461dc5e6622fc004b74044b7ff1b38.
ATTACHMENT ID: 12968636
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
+1 core tests. The patch passed unit tests in .
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2578//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2578//console
This message is automatically generated.
I think grant and revoke is pretty rare so is not going to be an issue. You can revisit it if it becomes a problem. Checking permissions of course need to use the AccessChecker to make use of the cache. AccessChecker also does not provide an API for taking grant or revoke actions. Sounds like the latest patch takes a reasonable strategy.
LGTM, Thanks for the patch kiran.maturi! Please file a JIRA to remove the other usages of AccessControlClient. mihir6692 do you have any cycles to commit this to the 4.14, 4.x and master branches?
kiran.maturi Can you please attach patches that can be applied to the 4.x-HBase-1.4, 4.x-HBase-1.5 and master branches?
tdsilva thanks for the review. will upload the patches for 4.x-HBase1.4, 4.x-HBase-1.5 and master branches
FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-1.5 #2 (See https://builds.apache.org/job/Phoenix-4.x-HBase-1.5/2/)
PHOENIX-5269 PhoenixAccessController should use AccessChecker instead of (mihir6692: rev 3b92ec3f7fbbde03618a7cbea5e88b1a528ef6bc)
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/PermissionsCacheIT.java
Committed patches for 4.x-HBase-1.4/1.5 branches. kiran.maturi tdsilva
After commit build of Jenkins is failing for PHOENIX-5269 for 4.x-HBase-1.5 branch. tdsilva Do you have any idea why it's not able to pull hbase:1.5.0-SNAPSHOT dependency is not able to pull? I can see this dependency in previous build but not available in maven repo.
Jenkins is failing for
PHOENIX-5269for 4.x-HBase-1.5 branch. tdsilva Do you have any idea why it's not able to pull hbase:1.5.0-SNAPSHOT dependency is not able to pull? I can see this dependency in previous build but not available in maven repo.
We probably have to add a repository entry for the Apache snapshots repo to the POM while using the SNAPSHOT version. This would be an oversight made when creating the 1.5 branch. I will fix it on a new issue
FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-1.4 #147 (See https://builds.apache.org/job/Phoenix-4.x-HBase-1.4/147/)
PHOENIX-5269 PhoenixAccessController should use AccessChecker instead of (monani.mihir: rev f8ee79b0dca210884f3c6a441a44326240bc864c)
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/PermissionsCacheIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java
- (edit) pom.xml
kiran.maturi Does this patch need to be committed to master as well?
- Why is the PermissionsCacheIT different between 4.x-HBase-1.4 and 4.x-HBase-1.5? (Test class has a parameter in 1.4 but not 1.5)
- The PermissionsCacheIT has been failing since this change in 4.x-HBase-1.4 and nobody noticed And it fails exactly because there's no no-parameter constructor.
- Why is this not needed in 4.x-HBase-1.3?
- Where's the master patch?
This is confusing on many fronts.
Please do not 1/2 check-in changes. Now we have a 1/2 open jira with changes in some branches and not in others, we can now neither close this jira, nor can we leave it open.
In 4.x-HBase-1.4 let's either fix the test or revert.
I pushed an addendum to -1.4 to fix the test (make like the -1.5 version).
ABORTED: Integrated in Jenkins build Phoenix-4.x-HBase-1.4 #165 (See https://builds.apache.org/job/Phoenix-4.x-HBase-1.4/165/)
PHOENIX-5269 addendum: Fix PermissionsCacheIT. (larsh: rev 23f5fb5112b7446033b6dcc6e8930c7f344cbd83)
- (edit) phoenix-core/src/it/java/org/apache/phoenix/end2end/PermissionsCacheIT.java
lhofhansl my bad, I wanted to include it this patch in the 4.14.2 release as it caused a pretty large perf penalty. kiran.maturi was out of office for a while and is back now, hopefully we should be getting a patch soon.
Thanks for fixing the test failure!
edit: lhofhansl this is also needed for the HBase-1.3 branch, but depended on HBase 1.3.5 being released. kiran.maturi now that HBase 1.3.5 has been released can you also upload a patch for the 1.3 branch?
lhofhansl tdsilva sorry for the delay in the patch for the master. I am working on fixing the following test failure when i run the PermissionsCacheIT.
BasePermissionsIT#testAutomaticGrantWithIndexAndView() . I have uploaded the master v1 patch.
java.lang.AssertionError: Expected exception was not thrown for user 'unprivilegedUser_N000007' at org.junit.Assert.fail(Assert.java:88) at org.apache.phoenix.end2end.BasePermissionsIT.verifyDenied(BasePermissionsIT.java:726) at org.apache.phoenix.end2end.BasePermissionsIT.verifyDenied(BasePermissionsIT.java:717) at org.apache.phoenix.end2end.BasePermissionsIT.testAutomaticGrantWithIndexAndView(BasePermissionsIT.java:1164) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:50) at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12) at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:47) at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:325) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:78) at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:57) at org.junit.runners.ParentRunner$3.run(ParentRunner.java:290) at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:71) at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:288) at org.junit.runners.ParentRunner.access$000(ParentRunner.java:58) at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:268) at org.junit.internal.runners.statements.RunBefores.evaluate(RunBefores.java:26) at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:48) at org.junit.rules.RunRules.evaluate(RunRules.java:20) at org.junit.runners.ParentRunner.run(ParentRunner.java:363) at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:86) at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:38) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:538) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:760) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:460) at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:206)
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12971347/PHOENIX-5269.master.v1.patch
against master branch at commit 0e60a901a0615354c3ca021e395959aa530f6e06.
ATTACHMENT ID: 12971347
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 release audit. The applied patch generated 2 release audit warnings (more than the master's current 0 warnings).
-1 lineLengths. The patch introduces the following lines longer than 100:
+ if(cp.getClass().getName().equals(org.apache.hadoop.hbase.security.access.AccessController.class.getName())) {
+ ConnectionFactory.createConnection(((CoprocessorEnvironment) env).getConfiguration())) {
+ getUserPermsFromUserDefinedAccessController(userPermissions, connection,
-1 core tests. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PermissionNSEnabledIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PermissionsCacheIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.PermissionNSDisabledIT
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2643//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2643//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2643//console
This message is automatically generated.
tdsilva I need some help for the master patch. tests are failing as the zookeeper watcher is not being initialized.
PhoenixMetaDataControllerEnvironment in master branch implements CoprocessorEnvironment github link
public static class PhoenixMetaDataControllerEnvironment extends BaseEnvironment<PhoenixCoprocessor> implements CoprocessorEnvironment<PhoenixCoprocessor> {
For 4.x-HBase-1.5 branch PhoenixMetaDataControllerEnvironment implements RegionCoprocessorEnvironment github link which provided the RegionServerServices to get the zookeeper
public static class PhoenixMetaDataControllerEnvironment extends CoprocessorHost.Environment implements RegionCoprocessorEnvironment {
For the master branch PhoenixMetaDataControllerEnvironment provides an instance of RegionCoprocessorHost
PhoenixMetaDataControllerEnvironment.getCoprocessorHost()
I am not sure if CoprocessorHost#checkAndLoadInstance() can be used to get an instance of HasRegionServices/RegionServerServices to get the zookeeper. please suggest
kiran.maturi if you add a getRegionServerServices() method to PhoenixMetaDataControllerEnvironment and then in PhoenixAccessController check if the env is an instanceof PhoenixMetaDataControllerEnvironment and then call getRegionServerServices() would that work?
kiran.maturi I was able to get the test to pass by adding the @CoreCoprocessor annotation to MetadataEndpointImpl and also adding a getRegionCoprocessorEnvironment() method to PhoenixMetaDataControllerEnvironment. I have attached a diff of my changes.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12971787/diff.patch
against master branch at commit 5bf49e62601e7d241eb0f649b78253708b1ee42d.
ATTACHMENT ID: 12971787
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
-1 patch. The patch command could not apply the patch.
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2675//console
This message is automatically generated.
tdsilva thanks a lot for helping with that. Will make the changes upload the patch for master and branch1.3 as well
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12971991/PHOENIX-5269.master.v2.patch
against master branch at commit 85ba3e973f45f62ddfb67d484415d31860799200.
ATTACHMENT ID: 12971991
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 release audit. The applied patch generated 2 release audit warnings (more than the master's current 0 warnings).
-1 lineLengths. The patch introduces the following lines longer than 100:
+ if(cp.getClass().getName().equals(org.apache.hadoop.hbase.security.access.AccessController.class.getName())) {
+ ConnectionFactory.createConnection(((CoprocessorEnvironment) env).getConfiguration())) {
+ getUserPermsFromUserDefinedAccessController(userPermissions, connection,
-1 core tests. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterTableWithViewsIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.index.IndexMetadataIT
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2681//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2681//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2681//console
This message is automatically generated.
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12972062/PHOENIX-5269.4.x-HBase-1.3.v1.patch
against 4.x-HBase-1.3 branch at commit 56e0f0381fe30005f3cdd1718f99f3205e56d99e.
ATTACHMENT ID: 12972062
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
-1 release audit. The applied patch generated 2 release audit warnings (more than the master's current 0 warnings).
+1 lineLengths. The patch does not introduce lines longer than 100
-1 core tests. The patch failed these unit tests:
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.execute.PartialCommitIT
./phoenix-core/target/failsafe-reports/TEST-org.apache.phoenix.end2end.AlterSessionIT
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2683//testReport/
Release audit warnings: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2683//artifact/patchprocess/patchReleaseAuditWarnings.txt
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2683//console
This message is automatically generated.
FAILURE: Integrated in Jenkins build Phoenix-4.x-HBase-1.3 #441 (See https://builds.apache.org/job/Phoenix-4.x-HBase-1.3/441/)
PHOENIX-5269 use AccessChecker to check for user permisssions (tdsilva: rev eb8ac33029cd1ce781bf2f8b826502f642f735c5)
- (edit) pom.xml
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/PermissionsCacheIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java
FAILURE: Integrated in Jenkins build PreCommit-PHOENIX-Build #2687 (See https://builds.apache.org/job/PreCommit-PHOENIX-Build/2687/)
PHOENIX-5269 use AccessChecker to check for user permisssions (tdsilva: rev 1f2508dbde365aaedac628c89df237e8b6b46df8)
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixAccessController.java
- (add) phoenix-core/src/it/java/org/apache/phoenix/end2end/PermissionsCacheIT.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/MetaDataEndpointImpl.java
- (edit) phoenix-core/src/main/java/org/apache/phoenix/coprocessor/PhoenixMetaDataCoprocessorHost.java
-1 overall. Here are the results of testing the latest attachment
http://issues.apache.org/jira/secure/attachment/12972281/PHOENIX-5269.4.14-HBase-1.3.v1.patch
against 4.14-HBase-1.3 branch at commit 1f2508dbde365aaedac628c89df237e8b6b46df8.
ATTACHMENT ID: 12972281
+1 @author. The patch does not contain any @author tags.
-1 tests included. The patch doesn't appear to include any new or modified tests.
Please justify why no new tests are needed for this patch.
Also please list what manual steps were performed to verify this patch.
+1 javac. The applied patch does not increase the total number of javac compiler warnings.
+1 release audit. The applied patch does not increase the total number of release audit warnings.
+1 lineLengths. The patch does not introduce lines longer than 100
-1 core tests. The patch failed these unit tests:
org.apache.phoenix.hbase.index.covered.TestCoveredColumnIndexCodec
Test results: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2690//testReport/
Console output: https://builds.apache.org/job/PreCommit-PHOENIX-Build/2690//console
This message is automatically generated.
kiran.maturi Thanks for fixing this, committed to 4.14, 4.x and master branches.
Are you going to want to fix this in the 1.3 branches? kiran.maturi tdsilva If so you'll need an HBase JIRA for backporting AccessChecker to branch-1.3.