Uploaded image for project: 'Zeppelin'
  1. Zeppelin
  2. ZEPPELIN-1472

Create new LdapRealm based on Apache Knox LdapRealm Class

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.6.1
    • Fix Version/s: 0.7.0
    • Component/s: None
    • Labels:
      None

      Description

      In our environment we attempted to use the ActiveDirectoryGroupRealm and the LdapGroupRealm but unfortunately those implementations against Shiro do not support ADLDAP Global Catalog. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated.. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's per the forest. I have attached a semi-working modified KnoxLdapRealm which works against samAccountName and global catalog for auth.

      http://windowsitpro.com/active-directory/q-does-samaccountname-object-have-be-unique-active-directory-domain-or-entire-fores

      https://jorgequestforknowledge.wordpress.com/2010/10/12/user-principal-names-in-ad-part-1/

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 opened a pull request:

          https://github.com/apache/zeppelin/pull/1493

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm

              1. What is this PR for?
                Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          Information about samAccountName and userPrincipalName with ActiveDirectory
          http://windowsitpro.com/active-directory/q-does-samaccountname-object-have-be-unique-active-directory-domain-or-entire-fores
          https://jorgequestforknowledge.wordpress.com/2010/10/12/user-principal-names-in-ad-part-1/

              1. What type of PR is it?
                Improvement
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                shiro.ini
                [main]
                ldapRealm = org.apache.zeppelin.server.LdapRealm
                ldapRealm.contextFactory.systemUsername = CN=hdpbind,OU=Svc,DC=exadc,DC=w2k,DC=example,DC=com
                ldapRealm.contextFactory.systemPassword = ldapPassword
                ldapRealm.searchBase = dc=w2k,dc=example,dc=com
                ldapRealm.userSearchBase = dc=w2k,dc=example,dc=com
                ldapRealm.groupSearchBase = dc=w2k,dc=example,dc=com
                ldapRealm.contextFactory.url = ldap://exampledc1.exadc.w2k.example.com:3268
                ldapRealm.userSearchAttributeName = sAMAccountName
                ldapRealm.contextFactory.authenticationMechanism = simple
                ldapRealm.userObjectClass = user
                ldapRealm.groupObjectClass = group
                ldapRealm.memberAttribute = member
                securityManager.realms = $ldapRealm
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin master

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1493.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1493


          commit 4b5963a2019f1fded13e6ce9942033101ef2acf1
          Author: Initial Commit <gsenia@apache.org>
          Date: 2016-10-07T00:55:42Z

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm
          Class

          In our environment we attempted to use the ActiveDirectoryGroupRealm and
          the LdapGroupRealm but unfortunately those implementations against Shiro
          do not support ADLDAP Global Catalog. Also searching on
          "userPrincipalName" is risky in an AD environment since the explicit UPN
          vs Implicit UPN can be different. And the LDAP userPrincipalName
          attribute is the explicit UPN which can be defined by the directory
          administrator to any value and it can be duplicated.. SamAccountName is
          unique per domain and Microsoft states best practice is to not allow
          duplicate samAccountName's per the forest. I have attached a
          semi-working modified KnoxLdapRealm which works against samAccountName
          and global catalog for auth.
          http://windowsitpro.com/active-directory/q-does-samaccountname-object-have-be-unique-active-directory-domain-or-entire-fores
          https://jorgequestforknowledge.wordpress.com/2010/10/12/user-principal-names-in-ad-part-1/


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 opened a pull request: https://github.com/apache/zeppelin/pull/1493 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm What is this PR for? Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. Information about samAccountName and userPrincipalName with ActiveDirectory http://windowsitpro.com/active-directory/q-does-samaccountname-object-have-be-unique-active-directory-domain-or-entire-fores https://jorgequestforknowledge.wordpress.com/2010/10/12/user-principal-names-in-ad-part-1/ What type of PR is it? Improvement What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? shiro.ini [main] ldapRealm = org.apache.zeppelin.server.LdapRealm ldapRealm.contextFactory.systemUsername = CN=hdpbind,OU=Svc,DC=exadc,DC=w2k,DC=example,DC=com ldapRealm.contextFactory.systemPassword = ldapPassword ldapRealm.searchBase = dc=w2k,dc=example,dc=com ldapRealm.userSearchBase = dc=w2k,dc=example,dc=com ldapRealm.groupSearchBase = dc=w2k,dc=example,dc=com ldapRealm.contextFactory.url = ldap://exampledc1.exadc.w2k.example.com:3268 ldapRealm.userSearchAttributeName = sAMAccountName ldapRealm.contextFactory.authenticationMechanism = simple ldapRealm.userObjectClass = user ldapRealm.groupObjectClass = group ldapRealm.memberAttribute = member securityManager.realms = $ldapRealm Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1493.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1493 commit 4b5963a2019f1fded13e6ce9942033101ef2acf1 Author: Initial Commit <gsenia@apache.org> Date: 2016-10-07T00:55:42Z ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm Class In our environment we attempted to use the ActiveDirectoryGroupRealm and the LdapGroupRealm but unfortunately those implementations against Shiro do not support ADLDAP Global Catalog. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated.. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's per the forest. I have attached a semi-working modified KnoxLdapRealm which works against samAccountName and global catalog for auth. http://windowsitpro.com/active-directory/q-does-samaccountname-object-have-be-unique-active-directory-domain-or-entire-fores https://jorgequestforknowledge.wordpress.com/2010/10/12/user-principal-names-in-ad-part-1/
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1493

          @vinayshukla here is the pull request for the LdapRealm enhancement we discussed last week at HadoopWorld/Strata

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1493 @vinayshukla here is the pull request for the LdapRealm enhancement we discussed last week at HadoopWorld/Strata
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user khalidhuseynov commented on the issue:

          https://github.com/apache/zeppelin/pull/1493

          i believe there's also `org.apache.zeppelin.realm` package under `zeppelin` and in the long run maybe need to move these realms there rather than keeping under `server`

          Show
          githubbot ASF GitHub Bot added a comment - Github user khalidhuseynov commented on the issue: https://github.com/apache/zeppelin/pull/1493 i believe there's also `org.apache.zeppelin.realm` package under `zeppelin` and in the long run maybe need to move these realms there rather than keeping under `server`
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1493

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1493
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1493

          @khalidhuseynov
          I'm going to make the changes and move this under realms. I will also close this pull request and open a new one

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1493 @khalidhuseynov I'm going to make the changes and move this under realms. I will also close this pull request and open a new one
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 opened a pull request:

          https://github.com/apache/zeppelin/pull/1513

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.
              1. What type of PR is it?
                [Improvement]
              1. Todos
                None
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Setup shiro.ini to use the following configuration:
                ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
                ldapADGCRealm.contextFactory.systemUsername = CN=hdpbind,OU=Svc,DC=exadc,DC=w2k,DC=example,DC=com
                ldapADGCRealm.contextFactory.systemPassword = ldapPassword
                ldapADGCRealm.searchBase = dc=w2k,dc=example,dc=com
                ldapADGCRealm.userSearchBase = dc=w2k,dc=example,dc=com
                ldapADGCRealm.groupSearchBase = dc=w2k,dc=example,dc=com
                ldapADGCRealm.contextFactory.url = ldap://exampledc1.exadc.w2k.example.com:3268
                ldapADGCRealm.userSearchAttributeName = sAMAccountName
                ldapADGCRealm.contextFactory.authenticationMechanism = simple
                ldapADGCRealm.userObjectClass = user
                ldapADGCRealm.groupObjectClass = group
                ldapADGCRealm.memberAttribute = member
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? n

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1513.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1513


          commit 34938754ac7e220a03cc1817bf93f2cf2d189ee9
          Author: gss2002 <greg@senia.org>
          Date: 2016-10-11T03:58:51Z

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm
          Class

          commit 8991d647b024d04eed7005173b4a8eec07b18c6c
          Author: gss2002 <greg@senia.org>
          Date: 2016-10-14T00:48:25Z

          Merge remote-tracking branch 'upstream/master' into ZEPPELIN-1472


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 opened a pull request: https://github.com/apache/zeppelin/pull/1513 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. What type of PR is it? [Improvement] Todos None What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Setup shiro.ini to use the following configuration: ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdpbind,OU=Svc,DC=exadc,DC=w2k,DC=example,DC=com ldapADGCRealm.contextFactory.systemPassword = ldapPassword ldapADGCRealm.searchBase = dc=w2k,dc=example,dc=com ldapADGCRealm.userSearchBase = dc=w2k,dc=example,dc=com ldapADGCRealm.groupSearchBase = dc=w2k,dc=example,dc=com ldapADGCRealm.contextFactory.url = ldap://exampledc1.exadc.w2k.example.com:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.userObjectClass = user ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? n You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1513.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1513 commit 34938754ac7e220a03cc1817bf93f2cf2d189ee9 Author: gss2002 <greg@senia.org> Date: 2016-10-11T03:58:51Z ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm Class commit 8991d647b024d04eed7005173b4a8eec07b18c6c Author: gss2002 <greg@senia.org> Date: 2016-10-14T00:48:25Z Merge remote-tracking branch 'upstream/master' into ZEPPELIN-1472
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @khalidhuseynov made the requested changes and updated documentation. Please let me know what you think. Also I will be willing to create a jira and move the other Realms.

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1513 @khalidhuseynov made the requested changes and updated documentation. Please let me know what you think. Also I will be willing to create a jira and move the other Realms.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1513

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1513
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          Rerun build as error is not related to this patch.

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1513 Rerun build as error is not related to this patch.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 reopened a pull request:

          https://github.com/apache/zeppelin/pull/1513

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.
              1. What type of PR is it?
                [Improvement]
              1. Todos
                None
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Setup shiro.ini to use the following configuration:
                ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
                ldapADGCRealm.contextFactory.systemUsername = CN=hdpbind,OU=Svc,DC=exadc,DC=w2k,DC=example,DC=com
                ldapADGCRealm.contextFactory.systemPassword = ldapPassword
                ldapADGCRealm.searchBase = dc=w2k,dc=example,dc=com
                ldapADGCRealm.userSearchBase = dc=w2k,dc=example,dc=com
                ldapADGCRealm.groupSearchBase = dc=w2k,dc=example,dc=com
                ldapADGCRealm.contextFactory.url = ldap://exampledc1.exadc.w2k.example.com:3268
                ldapADGCRealm.userSearchAttributeName = sAMAccountName
                ldapADGCRealm.contextFactory.authenticationMechanism = simple
                ldapADGCRealm.userObjectClass = user
                ldapADGCRealm.groupObjectClass = group
                ldapADGCRealm.memberAttribute = member
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? n

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1513.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1513


          commit 34938754ac7e220a03cc1817bf93f2cf2d189ee9
          Author: gss2002 <greg@senia.org>
          Date: 2016-10-11T03:58:51Z

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm
          Class

          commit 8991d647b024d04eed7005173b4a8eec07b18c6c
          Author: gss2002 <greg@senia.org>
          Date: 2016-10-14T00:48:25Z

          Merge remote-tracking branch 'upstream/master' into ZEPPELIN-1472


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 reopened a pull request: https://github.com/apache/zeppelin/pull/1513 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. What type of PR is it? [Improvement] Todos None What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Setup shiro.ini to use the following configuration: ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdpbind,OU=Svc,DC=exadc,DC=w2k,DC=example,DC=com ldapADGCRealm.contextFactory.systemPassword = ldapPassword ldapADGCRealm.searchBase = dc=w2k,dc=example,dc=com ldapADGCRealm.userSearchBase = dc=w2k,dc=example,dc=com ldapADGCRealm.groupSearchBase = dc=w2k,dc=example,dc=com ldapADGCRealm.contextFactory.url = ldap://exampledc1.exadc.w2k.example.com:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.userObjectClass = user ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? n You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1513.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1513 commit 34938754ac7e220a03cc1817bf93f2cf2d189ee9 Author: gss2002 <greg@senia.org> Date: 2016-10-11T03:58:51Z ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm Class commit 8991d647b024d04eed7005173b4a8eec07b18c6c Author: gss2002 <greg@senia.org> Date: 2016-10-14T00:48:25Z Merge remote-tracking branch 'upstream/master' into ZEPPELIN-1472
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @khalidhuseynov and @zjffdu can we look at committing this since tests have passed. If not let me know what else is needed.

          Thanks

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1513 @khalidhuseynov and @zjffdu can we look at committing this since tests have passed. If not let me know what else is needed. Thanks
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jongyoul commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @gss2002 Hi, Thanks for the contribution. This PR actually has no problem itself or no breaking current behavior. So LGTM. But I'm just curious about the use case. Do you know the case to use this? May be in your company?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1513 @gss2002 Hi, Thanks for the contribution. This PR actually has no problem itself or no breaking current behavior. So LGTM. But I'm just curious about the use case. Do you know the case to use this? May be in your company?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jongyoul commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @nazgul33 I've heard your team used LDAP. Can you adopt and test this PR in your team?

          Show
          githubbot ASF GitHub Bot added a comment - Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1513 @nazgul33 I've heard your team used LDAP. Can you adopt and test this PR in your team?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user nazgul33 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @jongyoul I could make it work with openLDAP server. I had no time to play around with group settings, but at least I could authenticate user by

          • ldapADGCRealm.userSearchAttributeName = cn
          • ldapADGCRealm.userSearchAttributeName = uid

          and the followings are settings I had to modify when I'm using phpldapadmin to manage users.
          ldapADGCRealm.contextFactory.authenticationMechanism = simple
          ldapADGCRealm.userObjectClass = posixAccount
          ldapADGCRealm.groupObjectClass = posixGroup
          ldapADGCRealm.memberAttribute = memberUid

          for me, it works.

          Show
          githubbot ASF GitHub Bot added a comment - Github user nazgul33 commented on the issue: https://github.com/apache/zeppelin/pull/1513 @jongyoul I could make it work with openLDAP server. I had no time to play around with group settings, but at least I could authenticate user by ldapADGCRealm.userSearchAttributeName = cn ldapADGCRealm.userSearchAttributeName = uid and the followings are settings I had to modify when I'm using phpldapadmin to manage users. ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.userObjectClass = posixAccount ldapADGCRealm.groupObjectClass = posixGroup ldapADGCRealm.memberAttribute = memberUid for me, it works.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @nazgul33 and @jongyoul this has been tested from a group perspective:

          Here is the example:
          [gsenia@hdp25sandbox ~]$ cat /etc/zeppelin/conf/shiro.ini

          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapRealm = org.apache.zeppelin.realm.LdapRealm
            ldapRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapRealm.contextFactory.systemPassword = ldapBind12
            ldapRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapRealm.authorizationEnabled = true
            ldapRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapRealm.userSearchAttributeName = sAMAccountName
            ldapRealm.contextFactory.authenticationMechanism = simple
            ldapRealm.groupObjectClass = group
            ldapRealm.memberAttribute = member
            ldapRealm.rolesByGroup = hdpeng: admin

          securityManager.realms = $ldapRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1513 @nazgul33 and @jongyoul this has been tested from a group perspective: Here is the example: [gsenia@hdp25sandbox ~] $ cat /etc/zeppelin/conf/shiro.ini Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapRealm = org.apache.zeppelin.realm.LdapRealm ldapRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapRealm.contextFactory.systemPassword = ldapBind12 ldapRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapRealm.authorizationEnabled = true ldapRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapRealm.userSearchAttributeName = sAMAccountName ldapRealm.contextFactory.authenticationMechanism = simple ldapRealm.groupObjectClass = group ldapRealm.memberAttribute = member ldapRealm.rolesByGroup = hdpeng: admin securityManager.realms = $ldapRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin]
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user nazgul33 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @gss2002 Thanks for the sample configuration.
          I'm going to test this PR further with group mappings on Monday.

          Show
          githubbot ASF GitHub Bot added a comment - Github user nazgul33 commented on the issue: https://github.com/apache/zeppelin/pull/1513 @gss2002 Thanks for the sample configuration. I'm going to test this PR further with group mappings on Monday.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user nazgul33 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @gss2002 can you explain how to map ldap groups to shiro roles?
          What i'm trying to do is to configure user rights to notebooks.

          ```
          shiro.ini


          ldapADGCRealm.userObjectClass = posixAccount
          ldapADGCRealm.groupObjectClass = posixGroup
          ldapADGCRealm.memberAttribute = memberUid
          ldapADGCRealm.rolesByGroup = \
          dataservice:admin, \
          woowabros:woowabros

          securityManager.realms = $ldapADGCRealm

          [roles]
          admin = *
          woowabros = *

          ```
          Added "testuser" in group "woowabros" by setting memberUid attribute of woowabros.
          if I access a notebook with security setup
          owner = nazgul33
          read = woowabros

          an error pops out like this

          > Insufficient privileges to read notebook.
          >
          > Allowed users or roles: [woowabros]
          >
          > But the user testuser belongs to: [testuser]

          Show
          githubbot ASF GitHub Bot added a comment - Github user nazgul33 commented on the issue: https://github.com/apache/zeppelin/pull/1513 @gss2002 can you explain how to map ldap groups to shiro roles? What i'm trying to do is to configure user rights to notebooks. ``` shiro.ini ldapADGCRealm.userObjectClass = posixAccount ldapADGCRealm.groupObjectClass = posixGroup ldapADGCRealm.memberAttribute = memberUid ldapADGCRealm.rolesByGroup = \ dataservice:admin, \ woowabros:woowabros securityManager.realms = $ldapADGCRealm [roles] admin = * woowabros = * ``` Added "testuser" in group "woowabros" by setting memberUid attribute of woowabros. if I access a notebook with security setup owner = nazgul33 read = woowabros an error pops out like this > Insufficient privileges to read notebook. > > Allowed users or roles: [woowabros] > > But the user testuser belongs to: [testuser]
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          @nazgul33 refactoring some code fix coming.. To utilize groups and users

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1513 @nazgul33 refactoring some code fix coming.. To utilize groups and users
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1513

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1513
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 opened a pull request:

          https://github.com/apache/zeppelin/pull/1614

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.

          The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.

          I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.

          Example - SecurityUtils
          String name = realm.getClass().getName();
          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { allRoles = ((IniRealm) realm).getIni().get("roles"); break; }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { allRoles = ((LdapRealm) realm).getListRoles(); break; }

          Example - SecurityRestApi:
          String name = realm.getClass().getName();
          if (LOG.isDebugEnabled())

          { LOG.debug("RealmClass.getName: " + name); }

          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); }

          else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm"))

          { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); }

          Please see feedback from previous PRs related to this JIRA:
          https://github.com/apache/zeppelin/pull/1513

              1. What type of PR is it?
                [Improvement]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Update shiro.ini to use configuration similar to below:
          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
            ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
            ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.authorizationEnabled = true
            ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapADGCRealm.userSearchAttributeName = sAMAccountName
            ldapADGCRealm.contextFactory.authenticationMechanism = simple
            ldapADGCRealm.groupObjectClass = group
            ldapADGCRealm.memberAttribute = member
            ldapADGCRealm.rolesByGroup = hdpeng: admin, \
            hadoopusers: user

          securityManager.realms = $ldapADGCRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *
            user = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
            /api/login = authc
            /api/login/logout = authc
            /api/security/ticket = authc
            /** = authc, roles[admin, user]
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          merge latest commits

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1614.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1614


          commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1
          Author: gss2002 <gsenia@apache.org>
          Date: 2016-11-08T16:26:11Z

          Merge pull request #1 from apache/master

          merge latest commits

          commit 097e66556c0d008d5d26e72ba998aa9cce079a36
          Author: gss2002 <greg@senia.org>
          Date: 2016-11-08T18:14:46Z

          ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support
          of using roles with LdapRealms. Also adjusted to use className and not
          actual name of the realm in shiro.ini. As using realmName in code could
          cause problems for people who want to use alternate names. Also migrated
          the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm
          packages per a recommendation.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 opened a pull request: https://github.com/apache/zeppelin/pull/1614 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent. The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization. I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer. Example - SecurityUtils String name = realm.getClass().getName(); if (name.equals("org.apache.shiro.realm.text.IniRealm")) { allRoles = ((IniRealm) realm).getIni().get("roles"); break; } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { allRoles = ((LdapRealm) realm).getListRoles(); break; } Example - SecurityRestApi: String name = realm.getClass().getName(); if (LOG.isDebugEnabled()) { LOG.debug("RealmClass.getName: " + name); } if (name.equals("org.apache.shiro.realm.text.IniRealm")) { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); } else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); } Please see feedback from previous PRs related to this JIRA: https://github.com/apache/zeppelin/pull/1513 What type of PR is it? [Improvement] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Update shiro.ini to use configuration similar to below: Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member ldapADGCRealm.rolesByGroup = hdpeng: admin, \ hadoopusers: user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * user = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] /api/login = authc /api/login/logout = authc /api/security/ticket = authc /** = authc, roles [admin, user] Screenshots (if appropriate) Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y merge latest commits You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1614 commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1 Author: gss2002 <gsenia@apache.org> Date: 2016-11-08T16:26:11Z Merge pull request #1 from apache/master merge latest commits commit 097e66556c0d008d5d26e72ba998aa9cce079a36 Author: gss2002 <greg@senia.org> Date: 2016-11-08T18:14:46Z ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support of using roles with LdapRealms. Also adjusted to use className and not actual name of the realm in shiro.ini. As using realmName in code could cause problems for people who want to use alternate names. Also migrated the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm packages per a recommendation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1614

          @nazgul33 and @jongyoul I made some changes to support the groups as roles. let me know. So far so good in user testing at the enterprise I work for. This replaces #1513

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1614 @nazgul33 and @jongyoul I made some changes to support the groups as roles. let me know. So far so good in user testing at the enterprise I work for. This replaces #1513
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1513

          This is replaced by #1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1513 This is replaced by #1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1614

          reopen for clean up

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1614 reopen for clean up
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 reopened a pull request:

          https://github.com/apache/zeppelin/pull/1614

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.

          The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.

          I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.

          Example - SecurityUtils
          String name = realm.getClass().getName();
          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { allRoles = ((IniRealm) realm).getIni().get("roles"); break; }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { allRoles = ((LdapRealm) realm).getListRoles(); break; }

          Example - SecurityRestApi:
          String name = realm.getClass().getName();
          if (LOG.isDebugEnabled())

          { LOG.debug("RealmClass.getName: " + name); }

          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); }

          else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm"))

          { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); }

          Please see feedback from previous PRs related to this JIRA:
          https://github.com/apache/zeppelin/pull/1513

              1. What type of PR is it?
                [Improvement]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Update shiro.ini to use configuration similar to below:
          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
            ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
            ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.authorizationEnabled = true
            ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapADGCRealm.userSearchAttributeName = sAMAccountName
            ldapADGCRealm.contextFactory.authenticationMechanism = simple
            ldapADGCRealm.groupObjectClass = group
            ldapADGCRealm.memberAttribute = member
            ldapADGCRealm.rolesByGroup = hdpeng: admin, \
            hadoopusers: user

          securityManager.realms = $ldapADGCRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *
            user = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
            /api/login = authc
            /api/login/logout = authc
            /api/security/ticket = authc
            /** = authc, roles[admin, user]
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          merge latest commits

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1614.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1614


          commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1
          Author: gss2002 <gsenia@apache.org>
          Date: 2016-11-08T16:26:11Z

          Merge pull request #1 from apache/master

          merge latest commits

          commit 635deb3398fded9811c05caa688ba950ba7e8d1b
          Author: gss2002 <greg@senia.org>
          Date: 2016-11-08T18:14:46Z

          ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support
          of using roles with LdapRealms. Also adjusted to use className and not
          actual name of the realm in shiro.ini. As using realmName in code could
          cause problems for people who want to use alternate names. Also migrated
          the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm
          packages per a recommendation.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 reopened a pull request: https://github.com/apache/zeppelin/pull/1614 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent. The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization. I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer. Example - SecurityUtils String name = realm.getClass().getName(); if (name.equals("org.apache.shiro.realm.text.IniRealm")) { allRoles = ((IniRealm) realm).getIni().get("roles"); break; } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { allRoles = ((LdapRealm) realm).getListRoles(); break; } Example - SecurityRestApi: String name = realm.getClass().getName(); if (LOG.isDebugEnabled()) { LOG.debug("RealmClass.getName: " + name); } if (name.equals("org.apache.shiro.realm.text.IniRealm")) { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); } else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); } Please see feedback from previous PRs related to this JIRA: https://github.com/apache/zeppelin/pull/1513 What type of PR is it? [Improvement] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Update shiro.ini to use configuration similar to below: Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member ldapADGCRealm.rolesByGroup = hdpeng: admin, \ hadoopusers: user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * user = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] /api/login = authc /api/login/logout = authc /api/security/ticket = authc /** = authc, roles [admin, user] Screenshots (if appropriate) Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y merge latest commits You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1614 commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1 Author: gss2002 <gsenia@apache.org> Date: 2016-11-08T16:26:11Z Merge pull request #1 from apache/master merge latest commits commit 635deb3398fded9811c05caa688ba950ba7e8d1b Author: gss2002 <greg@senia.org> Date: 2016-11-08T18:14:46Z ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support of using roles with LdapRealms. Also adjusted to use className and not actual name of the realm in shiro.ini. As using realmName in code could cause problems for people who want to use alternate names. Also migrated the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm packages per a recommendation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 reopened a pull request:

          https://github.com/apache/zeppelin/pull/1614

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.

          The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.

          I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.

          Example - SecurityUtils
          String name = realm.getClass().getName();
          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { allRoles = ((IniRealm) realm).getIni().get("roles"); break; }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { allRoles = ((LdapRealm) realm).getListRoles(); break; }

          Example - SecurityRestApi:
          String name = realm.getClass().getName();
          if (LOG.isDebugEnabled())

          { LOG.debug("RealmClass.getName: " + name); }

          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); }

          else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm"))

          { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); }

          Please see feedback from previous PRs related to this JIRA:
          https://github.com/apache/zeppelin/pull/1513

              1. What type of PR is it?
                [Improvement]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Update shiro.ini to use configuration similar to below:
          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
            ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
            ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.authorizationEnabled = true
            ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapADGCRealm.userSearchAttributeName = sAMAccountName
            ldapADGCRealm.contextFactory.authenticationMechanism = simple
            ldapADGCRealm.groupObjectClass = group
            ldapADGCRealm.memberAttribute = member
            ldapADGCRealm.rolesByGroup = hdpeng: admin, \
            hadoopusers: user

          securityManager.realms = $ldapADGCRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *
            user = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
            /api/login = authc
            /api/login/logout = authc
            /api/security/ticket = authc
            /** = authc, roles[admin, user]
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          merge latest commits

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1614.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1614


          commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1
          Author: gss2002 <gsenia@apache.org>
          Date: 2016-11-08T16:26:11Z

          Merge pull request #1 from apache/master

          merge latest commits

          commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3
          Author: gss2002 <greg@senia.org>
          Date: 2016-11-08T18:14:46Z

          ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support
          of using roles with LdapRealms. Also adjusted to use className and not
          actual name of the realm in shiro.ini. As using realmName in code could
          cause problems for people who want to use alternate names. Also migrated
          the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm
          packages per a recommendation.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 reopened a pull request: https://github.com/apache/zeppelin/pull/1614 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent. The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization. I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer. Example - SecurityUtils String name = realm.getClass().getName(); if (name.equals("org.apache.shiro.realm.text.IniRealm")) { allRoles = ((IniRealm) realm).getIni().get("roles"); break; } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { allRoles = ((LdapRealm) realm).getListRoles(); break; } Example - SecurityRestApi: String name = realm.getClass().getName(); if (LOG.isDebugEnabled()) { LOG.debug("RealmClass.getName: " + name); } if (name.equals("org.apache.shiro.realm.text.IniRealm")) { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); } else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); } Please see feedback from previous PRs related to this JIRA: https://github.com/apache/zeppelin/pull/1513 What type of PR is it? [Improvement] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Update shiro.ini to use configuration similar to below: Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member ldapADGCRealm.rolesByGroup = hdpeng: admin, \ hadoopusers: user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * user = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] /api/login = authc /api/login/logout = authc /api/security/ticket = authc /** = authc, roles [admin, user] Screenshots (if appropriate) Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y merge latest commits You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1614 commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1 Author: gss2002 <gsenia@apache.org> Date: 2016-11-08T16:26:11Z Merge pull request #1 from apache/master merge latest commits commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3 Author: gss2002 <greg@senia.org> Date: 2016-11-08T18:14:46Z ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support of using roles with LdapRealms. Also adjusted to use className and not actual name of the realm in shiro.ini. As using realmName in code could cause problems for people who want to use alternate names. Also migrated the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm packages per a recommendation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 reopened a pull request:

          https://github.com/apache/zeppelin/pull/1614

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.

          The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.

          I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.

          Example - SecurityUtils
          String name = realm.getClass().getName();
          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { allRoles = ((IniRealm) realm).getIni().get("roles"); break; }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { allRoles = ((LdapRealm) realm).getListRoles(); break; }

          Example - SecurityRestApi:
          String name = realm.getClass().getName();
          if (LOG.isDebugEnabled())

          { LOG.debug("RealmClass.getName: " + name); }

          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); }

          else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm"))

          { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); }

          Please see feedback from previous PRs related to this JIRA:
          https://github.com/apache/zeppelin/pull/1513

              1. What type of PR is it?
                [Improvement]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Update shiro.ini to use configuration similar to below:
          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
            ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
            ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.authorizationEnabled = true
            ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapADGCRealm.userSearchAttributeName = sAMAccountName
            ldapADGCRealm.contextFactory.authenticationMechanism = simple
            ldapADGCRealm.groupObjectClass = group
            ldapADGCRealm.memberAttribute = member
            ldapADGCRealm.rolesByGroup = hdpeng: admin, \
            hadoopusers: user

          securityManager.realms = $ldapADGCRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *
            user = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
            /api/login = authc
            /api/login/logout = authc
            /api/security/ticket = authc
            /** = authc, roles[admin, user]
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          merge latest commits

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1614.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1614


          commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1
          Author: gss2002 <gsenia@apache.org>
          Date: 2016-11-08T16:26:11Z

          Merge pull request #1 from apache/master

          merge latest commits

          commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3
          Author: gss2002 <greg@senia.org>
          Date: 2016-11-08T18:14:46Z

          ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support
          of using roles with LdapRealms. Also adjusted to use className and not
          actual name of the realm in shiro.ini. As using realmName in code could
          cause problems for people who want to use alternate names. Also migrated
          the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm
          packages per a recommendation.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 reopened a pull request: https://github.com/apache/zeppelin/pull/1614 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent. The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization. I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer. Example - SecurityUtils String name = realm.getClass().getName(); if (name.equals("org.apache.shiro.realm.text.IniRealm")) { allRoles = ((IniRealm) realm).getIni().get("roles"); break; } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { allRoles = ((LdapRealm) realm).getListRoles(); break; } Example - SecurityRestApi: String name = realm.getClass().getName(); if (LOG.isDebugEnabled()) { LOG.debug("RealmClass.getName: " + name); } if (name.equals("org.apache.shiro.realm.text.IniRealm")) { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); } else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); } Please see feedback from previous PRs related to this JIRA: https://github.com/apache/zeppelin/pull/1513 What type of PR is it? [Improvement] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Update shiro.ini to use configuration similar to below: Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member ldapADGCRealm.rolesByGroup = hdpeng: admin, \ hadoopusers: user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * user = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] /api/login = authc /api/login/logout = authc /api/security/ticket = authc /** = authc, roles [admin, user] Screenshots (if appropriate) Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y merge latest commits You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1614 commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1 Author: gss2002 <gsenia@apache.org> Date: 2016-11-08T16:26:11Z Merge pull request #1 from apache/master merge latest commits commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3 Author: gss2002 <greg@senia.org> Date: 2016-11-08T18:14:46Z ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support of using roles with LdapRealms. Also adjusted to use className and not actual name of the realm in shiro.ini. As using realmName in code could cause problems for people who want to use alternate names. Also migrated the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm packages per a recommendation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user nazgul33 commented on the issue:

          https://github.com/apache/zeppelin/pull/1614

          @gss2002 @jongyoul
          I confirm that this works as expected.
          ldap group to shiro role mapping works flawlessly.
          here's my shiro setup with openldap + phpldapadmin

          ```
          [main]
          ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
          ldapADGCRealm.contextFactory.systemUsername = cn=zeppelin,ou=system,dc=example,DC=com
          ldapADGCRealm.contextFactory.systemPassword = 1some2Random3Password4
          ldapADGCRealm.userDnTemplate=cn=

          {0},ou=people,dc=example,DC=com
          ldapADGCRealm.searchBase = dc=example,DC=com
          ldapADGCRealm.userSearchBase = ou=people,dc=example,DC=com
          ldapADGCRealm.groupSearchBase = ou=groups,dc=example,DC=com
          ldapADGCRealm.contextFactory.url = ldap://127.0.0.1:389
          ldapADGCRealm.contextFactory.authenticationMechanism = simple
          ldapADGCRealm.userObjectClass = posixAccount
          ldapADGCRealm.groupObjectClass = posixGroup
          ldapADGCRealm.authorizationEnabled = true
          ldapADGCRealm.memberAttribute = memberUid
          ldapADGCRealm.memberAttributeValueTemplate=cn={0}

          ,ou=people,dc=example,DC=com
          ldapADGCRealm.rolesByGroup = AdminGroup:admin,UserGroup:user

          securityManager.realms = $ldapADGCRealm
          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

          [roles]
          admin = *
          user = *
          ```

          I added all people under "ou=people,dc=example,DC=com",
          groups under "ou=people,dc=example,DC=com".
          In each groups, added "memberUid" property and added users to this attribute.

          now I can configure user or/and role for each note.
          other permissions for interpreter settings or whatever works as expected.

          ```
          [urls]
          /api/interpreter/** = authc, roles[admin]
          /api/configurations/** = authc, roles[admin]
          /api/credential/** = authc, roles[admin]
          ```

          thank for nice patch!!

          Show
          githubbot ASF GitHub Bot added a comment - Github user nazgul33 commented on the issue: https://github.com/apache/zeppelin/pull/1614 @gss2002 @jongyoul I confirm that this works as expected. ldap group to shiro role mapping works flawlessly. here's my shiro setup with openldap + phpldapadmin ``` [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = cn=zeppelin,ou=system,dc=example,DC=com ldapADGCRealm.contextFactory.systemPassword = 1some2Random3Password4 ldapADGCRealm.userDnTemplate=cn= {0},ou=people,dc=example,DC=com ldapADGCRealm.searchBase = dc=example,DC=com ldapADGCRealm.userSearchBase = ou=people,dc=example,DC=com ldapADGCRealm.groupSearchBase = ou=groups,dc=example,DC=com ldapADGCRealm.contextFactory.url = ldap://127.0.0.1:389 ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.userObjectClass = posixAccount ldapADGCRealm.groupObjectClass = posixGroup ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.memberAttribute = memberUid ldapADGCRealm.memberAttributeValueTemplate=cn={0} ,ou=people,dc=example,DC=com ldapADGCRealm.rolesByGroup = AdminGroup:admin,UserGroup:user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager [roles] admin = * user = * ``` I added all people under "ou=people,dc=example,DC=com", groups under "ou=people,dc=example,DC=com". In each groups, added "memberUid" property and added users to this attribute. now I can configure user or/and role for each note. other permissions for interpreter settings or whatever works as expected. ``` [urls] /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] ``` thank for nice patch!!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1614

          No problem Let me know if you see any others or if you need me to adjust this code at all

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1614 No problem Let me know if you see any others or if you need me to adjust this code at all
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 reopened a pull request:

          https://github.com/apache/zeppelin/pull/1614

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.

          The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.

          I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.

          Example - SecurityUtils
          String name = realm.getClass().getName();
          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { allRoles = ((IniRealm) realm).getIni().get("roles"); break; }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { allRoles = ((LdapRealm) realm).getListRoles(); break; }

          Example - SecurityRestApi:
          String name = realm.getClass().getName();
          if (LOG.isDebugEnabled())

          { LOG.debug("RealmClass.getName: " + name); }

          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); }

          else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm"))

          { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); }

          Please see feedback from previous PRs related to this JIRA:
          https://github.com/apache/zeppelin/pull/1513

              1. What type of PR is it?
                [Improvement]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Update shiro.ini to use configuration similar to below:
          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
            ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
            ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.authorizationEnabled = true
            ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapADGCRealm.userSearchAttributeName = sAMAccountName
            ldapADGCRealm.contextFactory.authenticationMechanism = simple
            ldapADGCRealm.groupObjectClass = group
            ldapADGCRealm.memberAttribute = member
            ldapADGCRealm.rolesByGroup = hdpeng: admin, \
            hadoopusers: user

          securityManager.realms = $ldapADGCRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *
            user = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
            /api/login = authc
            /api/login/logout = authc
            /api/security/ticket = authc
            /** = authc, roles[admin, user]
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          merge latest commits

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1614.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1614



          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 reopened a pull request: https://github.com/apache/zeppelin/pull/1614 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent. The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization. I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer. Example - SecurityUtils String name = realm.getClass().getName(); if (name.equals("org.apache.shiro.realm.text.IniRealm")) { allRoles = ((IniRealm) realm).getIni().get("roles"); break; } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { allRoles = ((LdapRealm) realm).getListRoles(); break; } Example - SecurityRestApi: String name = realm.getClass().getName(); if (LOG.isDebugEnabled()) { LOG.debug("RealmClass.getName: " + name); } if (name.equals("org.apache.shiro.realm.text.IniRealm")) { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); } else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); } Please see feedback from previous PRs related to this JIRA: https://github.com/apache/zeppelin/pull/1513 What type of PR is it? [Improvement] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Update shiro.ini to use configuration similar to below: Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member ldapADGCRealm.rolesByGroup = hdpeng: admin, \ hadoopusers: user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * user = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] /api/login = authc /api/login/logout = authc /api/security/ticket = authc /** = authc, roles [admin, user] Screenshots (if appropriate) Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y merge latest commits You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jongyoul commented on the issue:

          https://github.com/apache/zeppelin/pull/1614

          Sorry for the delay. LGTM, merging if there's no more discussion.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1614 Sorry for the delay. LGTM, merging if there's no more discussion.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jongyoul commented on the issue:

          https://github.com/apache/zeppelin/pull/1614

          @cloverhearts Can you see the error of this PR? I think it's irrelevant but want to check it double.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jongyoul commented on the issue: https://github.com/apache/zeppelin/pull/1614 @cloverhearts Can you see the error of this PR? I think it's irrelevant but want to check it double.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 commented on the issue:

          https://github.com/apache/zeppelin/pull/1614

          Going to close and re-open to re kick the tests. These tests have been flaky I dont think the error is related to the patch...

          15:38:55,459 ERROR org.apache.zeppelin.AbstractZeppelinIT:136 - Exception in ParagraphActionsIT while testEditOnDoubleClick
          org.openqa.selenium.ElementNotVisibleException: Element is not currently visible and so may not be interacted with
          Command duration or timeout: 30.04 seconds
          Build info: version: '2.48.2', revision: '41bccdd10cf2c0560f637404c2d96164b67d9d67', time: '2015-10-09 13:08:06'
          System info: host: 'testing-docker-60ee1fc8-0996-4929-93bf-f3f4ab1d7d4e', ip: '172.17.0.8', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-47-generic', java.version: '1.7.0_76'
          Session ID: e568225a-5433-4a6e-b11a-85faf279113b
          Driver info: org.openqa.selenium.firefox.FirefoxDriver
          Capabilities [

          {platform=LINUX, acceptSslCerts=true, javascriptEnabled=true, cssSelectorsEnabled=true, databaseEnabled=true, browserName=firefox, handlesAlerts=true, nativeEvents=false, webStorageEnabled=true, rotatable=false, locationContextEnabled=true, applicationCacheEnabled=true, takesScreenshot=true, version=31.0}

          ]
          at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
          at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)
          at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
          at java.lang.reflect.Constructor.newInstance(Constructor.java:526)
          at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206)
          at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158)
          at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:647)
          at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:326)
          at org.openqa.selenium.remote.RemoteWebElement.sendKeys(RemoteWebElement.java:121)
          at org.apache.zeppelin.integration.ParagraphActionsIT.testEditOnDoubleClick(ParagraphActionsIT.java:443)
          at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 commented on the issue: https://github.com/apache/zeppelin/pull/1614 Going to close and re-open to re kick the tests. These tests have been flaky I dont think the error is related to the patch... 15:38:55,459 ERROR org.apache.zeppelin.AbstractZeppelinIT:136 - Exception in ParagraphActionsIT while testEditOnDoubleClick org.openqa.selenium.ElementNotVisibleException: Element is not currently visible and so may not be interacted with Command duration or timeout: 30.04 seconds Build info: version: '2.48.2', revision: '41bccdd10cf2c0560f637404c2d96164b67d9d67', time: '2015-10-09 13:08:06' System info: host: 'testing-docker-60ee1fc8-0996-4929-93bf-f3f4ab1d7d4e', ip: '172.17.0.8', os.name: 'Linux', os.arch: 'amd64', os.version: '4.4.0-47-generic', java.version: '1.7.0_76' Session ID: e568225a-5433-4a6e-b11a-85faf279113b Driver info: org.openqa.selenium.firefox.FirefoxDriver Capabilities [ {platform=LINUX, acceptSslCerts=true, javascriptEnabled=true, cssSelectorsEnabled=true, databaseEnabled=true, browserName=firefox, handlesAlerts=true, nativeEvents=false, webStorageEnabled=true, rotatable=false, locationContextEnabled=true, applicationCacheEnabled=true, takesScreenshot=true, version=31.0} ] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) at java.lang.reflect.Constructor.newInstance(Constructor.java:526) at org.openqa.selenium.remote.ErrorHandler.createThrowable(ErrorHandler.java:206) at org.openqa.selenium.remote.ErrorHandler.throwIfResponseFailed(ErrorHandler.java:158) at org.openqa.selenium.remote.RemoteWebDriver.execute(RemoteWebDriver.java:647) at org.openqa.selenium.remote.RemoteWebElement.execute(RemoteWebElement.java:326) at org.openqa.selenium.remote.RemoteWebElement.sendKeys(RemoteWebElement.java:121) at org.apache.zeppelin.integration.ParagraphActionsIT.testEditOnDoubleClick(ParagraphActionsIT.java:443) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 reopened a pull request:

          https://github.com/apache/zeppelin/pull/1614

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.

          The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.

          I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.

          Example - SecurityUtils
          String name = realm.getClass().getName();
          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { allRoles = ((IniRealm) realm).getIni().get("roles"); break; }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { allRoles = ((LdapRealm) realm).getListRoles(); break; }

          Example - SecurityRestApi:
          String name = realm.getClass().getName();
          if (LOG.isDebugEnabled())

          { LOG.debug("RealmClass.getName: " + name); }

          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); }

          else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm"))

          { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); }

          Please see feedback from previous PRs related to this JIRA:
          https://github.com/apache/zeppelin/pull/1513

              1. What type of PR is it?
                [Improvement]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Update shiro.ini to use configuration similar to below:
          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
            ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
            ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.authorizationEnabled = true
            ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapADGCRealm.userSearchAttributeName = sAMAccountName
            ldapADGCRealm.contextFactory.authenticationMechanism = simple
            ldapADGCRealm.groupObjectClass = group
            ldapADGCRealm.memberAttribute = member
            ldapADGCRealm.rolesByGroup = hdpeng: admin, \
            hadoopusers: user

          securityManager.realms = $ldapADGCRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *
            user = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
            /api/login = authc
            /api/login/logout = authc
            /api/security/ticket = authc
            /** = authc, roles[admin, user]
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          merge latest commits

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1614.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1614


          commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1
          Author: gss2002 <gsenia@apache.org>
          Date: 2016-11-08T16:26:11Z

          Merge pull request #1 from apache/master

          merge latest commits

          commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3
          Author: gss2002 <greg@senia.org>
          Date: 2016-11-08T18:14:46Z

          ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support
          of using roles with LdapRealms. Also adjusted to use className and not
          actual name of the realm in shiro.ini. As using realmName in code could
          cause problems for people who want to use alternate names. Also migrated
          the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm
          packages per a recommendation.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 reopened a pull request: https://github.com/apache/zeppelin/pull/1614 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent. The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization. I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer. Example - SecurityUtils String name = realm.getClass().getName(); if (name.equals("org.apache.shiro.realm.text.IniRealm")) { allRoles = ((IniRealm) realm).getIni().get("roles"); break; } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { allRoles = ((LdapRealm) realm).getListRoles(); break; } Example - SecurityRestApi: String name = realm.getClass().getName(); if (LOG.isDebugEnabled()) { LOG.debug("RealmClass.getName: " + name); } if (name.equals("org.apache.shiro.realm.text.IniRealm")) { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); } else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); } Please see feedback from previous PRs related to this JIRA: https://github.com/apache/zeppelin/pull/1513 What type of PR is it? [Improvement] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Update shiro.ini to use configuration similar to below: Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member ldapADGCRealm.rolesByGroup = hdpeng: admin, \ hadoopusers: user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * user = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] /api/login = authc /api/login/logout = authc /api/security/ticket = authc /** = authc, roles [admin, user] Screenshots (if appropriate) Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y merge latest commits You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1614 commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1 Author: gss2002 <gsenia@apache.org> Date: 2016-11-08T16:26:11Z Merge pull request #1 from apache/master merge latest commits commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3 Author: gss2002 <greg@senia.org> Date: 2016-11-08T18:14:46Z ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support of using roles with LdapRealms. Also adjusted to use className and not actual name of the realm in shiro.ini. As using realmName in code could cause problems for people who want to use alternate names. Also migrated the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm packages per a recommendation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user gss2002 closed the pull request at:

          https://github.com/apache/zeppelin/pull/1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user gss2002 closed the pull request at: https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user gss2002 reopened a pull request:

          https://github.com/apache/zeppelin/pull/1614

          ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox

              1. What is this PR for?
                ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest.

          In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent.

          The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization.

          I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer.

          Example - SecurityUtils
          String name = realm.getClass().getName();
          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { allRoles = ((IniRealm) realm).getIni().get("roles"); break; }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { allRoles = ((LdapRealm) realm).getListRoles(); break; }

          Example - SecurityRestApi:
          String name = realm.getClass().getName();
          if (LOG.isDebugEnabled())

          { LOG.debug("RealmClass.getName: " + name); }

          if (name.equals("org.apache.shiro.realm.text.IniRealm"))

          { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); }

          else if (name.equals("org.apache.zeppelin.realm.LdapRealm"))

          { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); }

          else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm"))

          { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); }

          else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm"))

          { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); }

          Please see feedback from previous PRs related to this JIRA:
          https://github.com/apache/zeppelin/pull/1513

              1. What type of PR is it?
                [Improvement]
              1. Todos
          • [ ] - Task
              1. What is the Jira issue?
                https://issues.apache.org/jira/browse/ZEPPELIN-1472
              1. How should this be tested?
                Update shiro.ini to use configuration similar to below:
          1. Sample LDAP configuration, for user Authentication, currently tested for single Realm
            [main]
            ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm
            ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org
            ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword
            ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org
            ldapADGCRealm.authorizationEnabled = true
            ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268
            ldapADGCRealm.userSearchAttributeName = sAMAccountName
            ldapADGCRealm.contextFactory.authenticationMechanism = simple
            ldapADGCRealm.groupObjectClass = group
            ldapADGCRealm.memberAttribute = member
            ldapADGCRealm.rolesByGroup = hdpeng: admin, \
            hadoopusers: user

          securityManager.realms = $ldapADGCRealm

          sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager

              1. If caching of user is required then uncomment below lines
                #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager
                #securityManager.cacheManager = $cacheManager

          securityManager.sessionManager = $sessionManager

          1. 86,400,000 milliseconds = 24 hour
            securityManager.sessionManager.globalSessionTimeout = 86400000
            shiro.loginUrl = /api/login

          [roles]

          1. 'admin' role has all permissions, indicated by the wildcard '*'
            admin = *
            user = *

          [urls]

          1. anon means the access is anonymous.
          2. authcBasic means Basic Auth Security
          3. authc means Form based Auth Security
          4. To enfore security, comment the line below and uncomment the next one
            #/api/version = anon
            #/** = anon
            /api/interpreter/** = authc, roles[admin]
            /api/configurations/** = authc, roles[admin]
            /api/credential/** = authc, roles[admin]
            /api/login = authc
            /api/login/logout = authc
            /api/security/ticket = authc
            /** = authc, roles[admin, user]
              1. Screenshots (if appropriate)
              1. Questions:
          • Does the licenses files need update? n
          • Is there breaking changes for older versions? n
          • Does this needs documentation? y

          merge latest commits

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/zeppelin/pull/1614.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1614


          commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1
          Author: gss2002 <gsenia@apache.org>
          Date: 2016-11-08T16:26:11Z

          Merge pull request #1 from apache/master

          merge latest commits

          commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3
          Author: gss2002 <greg@senia.org>
          Date: 2016-11-08T18:14:46Z

          ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support
          of using roles with LdapRealms. Also adjusted to use className and not
          actual name of the realm in shiro.ini. As using realmName in code could
          cause problems for people who want to use alternate names. Also migrated
          the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm
          packages per a recommendation.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user gss2002 reopened a pull request: https://github.com/apache/zeppelin/pull/1614 ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to Apache Knox What is this PR for? ZEPPELIN-1472 - Create new LdapRealm based on Apache Knox LdapRealm: Provides LdapRealm Functionality similar to what Apache Knox provides. This is critical as in large enterprise environments Active Directory Global Catalogs are used for lookup with samAccountName and using a DN Template is not an option as their are multiple OUs. Also searching on "userPrincipalName" is risky in an AD environment since the explicit UPN vs Implicit UPN can be different this is definitely the case with environments using Office 365. And the LDAP userPrincipalName attribute is the explicit UPN which can be defined by the directory administrator to any value and it can be duplicated. SamAccountName is unique per domain and Microsoft states best practice is to not allow duplicate samAccountName's across the forest. In addition to the above changes I have adjusted and moved the LdapGroupRealm and ActiveDirectoryGroupRealm into the org.apache.zeppelin.realm package structure to make all Realm's consistent. The LdapRealm class also works with role to group mapping for usage within Zeppelin for notebook authorization. I have adjusted SecurityUtils to use ClassName vs realmName in determining what to use as you may have companies that decide to use their own custom realmname in shiro.ini and may not realize you cannot so using className is much safer. Example - SecurityUtils String name = realm.getClass().getName(); if (name.equals("org.apache.shiro.realm.text.IniRealm")) { allRoles = ((IniRealm) realm).getIni().get("roles"); break; } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { allRoles = ((LdapRealm) realm).getListRoles(); break; } Example - SecurityRestApi: String name = realm.getClass().getName(); if (LOG.isDebugEnabled()) { LOG.debug("RealmClass.getName: " + name); } if (name.equals("org.apache.shiro.realm.text.IniRealm")) { usersList.addAll(getUserListObj.getUserList((IniRealm) realm)); rolesList.addAll(getUserListObj.getRolesList((IniRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.LdapGroupRealm")) { usersList.addAll(getUserListObj.getUserList((JndiLdapRealm) realm, searchText)); } else if (name.equals("org.apache.zeppelin.realm.LdapRealm")) { usersList.addAll(getUserListObj.getUserList((LdapRealm) realm, searchText)); rolesList.addAll(getUserListObj.getRolesList((LdapRealm) realm)); } else if (name.equals("org.apache.zeppelin.realm.ActiveDirectoryGroupRealm")) { usersList.addAll(getUserListObj.getUserList((ActiveDirectoryGroupRealm) realm, searchText)); } else if (name.equals("org.apache.shiro.realm.jdbc.JdbcRealm")) { usersList.addAll(getUserListObj.getUserList((JdbcRealm) realm)); } Please see feedback from previous PRs related to this JIRA: https://github.com/apache/zeppelin/pull/1513 What type of PR is it? [Improvement] Todos [ ] - Task What is the Jira issue? https://issues.apache.org/jira/browse/ZEPPELIN-1472 How should this be tested? Update shiro.ini to use configuration similar to below: Sample LDAP configuration, for user Authentication, currently tested for single Realm [main] ldapADGCRealm = org.apache.zeppelin.realm.LdapRealm ldapADGCRealm.contextFactory.systemUsername = CN=hdplookup,OU=hadoop,DC=hdpusr,DC=senia,DC=org ldapADGCRealm.contextFactory.systemPassword = ldapBindPassword ldapADGCRealm.searchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.userSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.groupSearchBase = dc=hdpusr,dc=senia,dc=org ldapADGCRealm.authorizationEnabled = true ldapADGCRealm.contextFactory.url = ldap://seniadc1.hdpusr.senia.org:3268 ldapADGCRealm.userSearchAttributeName = sAMAccountName ldapADGCRealm.contextFactory.authenticationMechanism = simple ldapADGCRealm.groupObjectClass = group ldapADGCRealm.memberAttribute = member ldapADGCRealm.rolesByGroup = hdpeng: admin, \ hadoopusers: user securityManager.realms = $ldapADGCRealm sessionManager = org.apache.shiro.web.session.mgt.DefaultWebSessionManager If caching of user is required then uncomment below lines #cacheManager = org.apache.shiro.cache.MemoryConstrainedCacheManager #securityManager.cacheManager = $cacheManager securityManager.sessionManager = $sessionManager 86,400,000 milliseconds = 24 hour securityManager.sessionManager.globalSessionTimeout = 86400000 shiro.loginUrl = /api/login [roles] 'admin' role has all permissions, indicated by the wildcard '*' admin = * user = * [urls] anon means the access is anonymous. authcBasic means Basic Auth Security authc means Form based Auth Security To enfore security, comment the line below and uncomment the next one #/api/version = anon #/** = anon /api/interpreter/** = authc, roles [admin] /api/configurations/** = authc, roles [admin] /api/credential/** = authc, roles [admin] /api/login = authc /api/login/logout = authc /api/security/ticket = authc /** = authc, roles [admin, user] Screenshots (if appropriate) Questions: Does the licenses files need update? n Is there breaking changes for older versions? n Does this needs documentation? y merge latest commits You can merge this pull request into a Git repository by running: $ git pull https://github.com/gss2002/zeppelin ZEPPELIN-1472 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/zeppelin/pull/1614.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1614 commit 1702cc52c23aa0e20bf1b11ebb3ecce279efe3a1 Author: gss2002 <gsenia@apache.org> Date: 2016-11-08T16:26:11Z Merge pull request #1 from apache/master merge latest commits commit d6a7cea1fe9555fc29f53a228ac1fbc320139ca3 Author: gss2002 <greg@senia.org> Date: 2016-11-08T18:14:46Z ZEPPELIN-1472 - LdapRealm Additions based on Knox LdapRealm and support of using roles with LdapRealms. Also adjusted to use className and not actual name of the realm in shiro.ini. As using realmName in code could cause problems for people who want to use alternate names. Also migrated the LdapGroupRealm and ActiveDirectoryRealm to org.apache.zeppelin.realm packages per a recommendation.
          Hide
          jongyoul Jongyoul Lee added a comment -

          Issue resolved by pull request 1614
          https://github.com/apache/zeppelin/pull/1614

          Show
          jongyoul Jongyoul Lee added a comment - Issue resolved by pull request 1614 https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/zeppelin/pull/1614

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/zeppelin/pull/1614
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bhavintandel commented on the issue:

          https://github.com/apache/zeppelin/pull/1493

          Do we have something like ldapADGCRealm.userLowerCase which will force username to change the case to lower. Currently if i set user permission to "user1" and then i login with "User1", i cannot see the notebooks.

          Show
          githubbot ASF GitHub Bot added a comment - Github user bhavintandel commented on the issue: https://github.com/apache/zeppelin/pull/1493 Do we have something like ldapADGCRealm.userLowerCase which will force username to change the case to lower. Currently if i set user permission to "user1" and then i login with "User1", i cannot see the notebooks.

            People

            • Assignee:
              gss2002 Greg Senia
              Reporter:
              gss2002 Greg Senia
            • Votes:
              0 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development