Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 5.3, 6.0
    • Component/s: None
    • Labels:
      None

      Description

      Relying on every Authentication plugin to secure the internode communication is error prone. Solr can standardize the authentication so that only the first request that comes from outside the cluster needs to be authenticated by the authentication plugin

      The scheme to protect the communication will be as follows

      • Every Solr node creates a an RSA key pair
      • The private key is kept private and the public key is made available through a core admin API
      • If authentication is enabled , every outgoing request will carry an extra header {{ SolrAuth : <nodename> encrypt_with_pvt_key(<original-user-principal> <timestamp>) }}
      • If authentication is enabled SolrDispatchFilter would look for this header and see the nodename
        • If the public key of the nodename is available in cache , make a request to the node and fetch the public key
        • If the public key has changed (because of a server restart) decryption fails and the public keyis fetched again
      • If the decryption succeeds , the user-name is set to what the header has encoded
      1. SOLR-7849.patch
        54 kB
        Noble Paul
      2. SOLR-7849.patch
        44 kB
        Noble Paul
      3. SOLR-7849.patch
        41 kB
        Noble Paul
      4. SOLR-7849.patch
        38 kB
        Noble Paul

        Issue Links

          Activity

          Hide
          Jan Høydahl added a comment -

          Interesting idea. How will node B be able to lookup the public key from core admin API of node A if A requires B to also authenticate? Perhaps publish pub-key through ZK instead of core admin?
          What should happen in multi-DC case; would cross cluster communication be treated as "internal"? What would <original-user-principal> be in case the action is initiated by Solr and not an external request?

          Show
          Jan Høydahl added a comment - Interesting idea. How will node B be able to lookup the public key from core admin API of node A if A requires B to also authenticate? Perhaps publish pub-key through ZK instead of core admin? What should happen in multi-DC case; would cross cluster communication be treated as "internal"? What would <original-user-principal> be in case the action is initiated by Solr and not an external request?
          Hide
          Noble Paul added a comment -

          How will node B be able to lookup the public key from core admin API of node A if A requires B to also authenticate? Perhaps publish pub-key through ZK instead of core admin?

          The public-key will be available at every node through a standard end-point e.g /admin/cores/key which will always be unprotected

          What should happen in multi-DC case; would cross cluster communication be treated as "internal"?

          That mechanism will have to be sorted out. Not a part of this ticket

          e.g : node-A in DC1 cluster wants to lookup node-P in DC2 cluster. We will publish the zk address of DC2 cluster in ZK of DC1 cluster and vice versa. This way node-A will trust al nodes in DC2 cluster as well

          What would <original-user-principal> be in case the action is initiated by Solr and not an external request?

          It will be a standard string like '$' which means the node itself is the principal

          Show
          Noble Paul added a comment - How will node B be able to lookup the public key from core admin API of node A if A requires B to also authenticate? Perhaps publish pub-key through ZK instead of core admin? The public-key will be available at every node through a standard end-point e.g /admin/cores/key which will always be unprotected What should happen in multi-DC case; would cross cluster communication be treated as "internal"? That mechanism will have to be sorted out. Not a part of this ticket e.g : node-A in DC1 cluster wants to lookup node-P in DC2 cluster. We will publish the zk address of DC2 cluster in ZK of DC1 cluster and vice versa. This way node-A will trust al nodes in DC2 cluster as well What would <original-user-principal> be in case the action is initiated by Solr and not an external request? It will be a standard string like '$' which means the node itself is the principal
          Hide
          Noble Paul added a comment -

          Implementation with a basic testcase

          Show
          Noble Paul added a comment - Implementation with a basic testcase
          Hide
          Noble Paul added a comment -

          more tests

          Show
          Noble Paul added a comment - more tests
          Hide
          Noble Paul added a comment -

          with proper integration test

          Show
          Noble Paul added a comment - with proper integration test
          Hide
          Ishan Chattopadhyaya added a comment - - edited

          +1, looks great!

          Here are some minor issues:
          1.

            private void addHttpConfigurer(Object authcPlugin) {
                ...
                log.info("Reconfiguring the shard handler factory and update shard handler.");
          

          Can we change it to:

                log.info("Reconfiguring the httpclients of shard handler factory and update shard handler.");
          

          2.

            private void addHttpConfigurer(Object authcPlugin) {
              log.info("addHttpConfigurer()");//TODO no commit
          

          I don't see any reason for the nocommit here. Is there something missing still?

          3.
          I think in SDF's `doFilter()`,

              if (cores.getAuthenticationPlugin() != null) {
          

          should be

              if (cores != null && cores.getAuthenticationPlugin() != null) {
          

          This is for those requests that come in even before the init() has finished execution.

          4. In PKIAuthenticationPlugin,

            private int maxValidity = 5000;
          

          Am I correct in my understanding that the difference of received internode request's timestamp and current timestamp shouldn't be more than this maxValidity? If that's true, I think an out of the box default of 5 secs is too less. Unless, we require all nodes to be using NTP; is that the plan? Also, shouldn't this be admin/user configurable?

          Show
          Ishan Chattopadhyaya added a comment - - edited +1, looks great! Here are some minor issues: 1. private void addHttpConfigurer(Object authcPlugin) { ... log.info("Reconfiguring the shard handler factory and update shard handler."); Can we change it to: log.info("Reconfiguring the httpclients of shard handler factory and update shard handler."); 2. private void addHttpConfigurer(Object authcPlugin) { log.info("addHttpConfigurer()");//TODO no commit I don't see any reason for the nocommit here. Is there something missing still? 3. I think in SDF's `doFilter()`, if (cores.getAuthenticationPlugin() != null) { should be if (cores != null && cores.getAuthenticationPlugin() != null) { This is for those requests that come in even before the init() has finished execution. 4. In PKIAuthenticationPlugin, private int maxValidity = 5000; Am I correct in my understanding that the difference of received internode request's timestamp and current timestamp shouldn't be more than this maxValidity? If that's true, I think an out of the box default of 5 secs is too less. Unless, we require all nodes to be using NTP; is that the plan? Also, shouldn't this be admin/user configurable?
          Hide
          ASF subversion and git services added a comment -

          Commit 1694217 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1694217 ]

          SOLR-7849: Solr-managed inter-node authentication when authentication enabled

          Show
          ASF subversion and git services added a comment - Commit 1694217 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1694217 ] SOLR-7849 : Solr-managed inter-node authentication when authentication enabled
          Hide
          ASF subversion and git services added a comment -

          Commit 1694239 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1694239 ]

          SOLR-7849: Solr-managed inter-node authentication when authentication enabled

          Show
          ASF subversion and git services added a comment - Commit 1694239 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1694239 ] SOLR-7849 : Solr-managed inter-node authentication when authentication enabled
          Hide
          Ishan Chattopadhyaya added a comment -

          There is still a "TODO: no commit" in CoreContainer.java.

          Show
          Ishan Chattopadhyaya added a comment - There is still a "TODO: no commit" in CoreContainer.java.
          Hide
          ASF subversion and git services added a comment -

          Commit 1694247 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1694247 ]

          SOLR-7849: Predicate is not available in java 7

          Show
          ASF subversion and git services added a comment - Commit 1694247 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1694247 ] SOLR-7849 : Predicate is not available in java 7
          Hide
          ASF subversion and git services added a comment -

          Commit 1694250 from Noble Paul in branch 'dev/branches/lucene_solr_5_3'
          [ https://svn.apache.org/r1694250 ]

          SOLR-7849: Predicate is not available in java 7

          Show
          ASF subversion and git services added a comment - Commit 1694250 from Noble Paul in branch 'dev/branches/lucene_solr_5_3' [ https://svn.apache.org/r1694250 ] SOLR-7849 : Predicate is not available in java 7
          Show
          Noble Paul added a comment - jenkins failures http://jenkins.thetaphi.de/job/Lucene-Solr-5.x-Linux/13572/
          Hide
          ASF subversion and git services added a comment -

          Commit 1694673 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1694673 ]

          SOLR-7849: Verify all nodes have received the new security config

          Show
          ASF subversion and git services added a comment - Commit 1694673 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1694673 ] SOLR-7849 : Verify all nodes have received the new security config
          Hide
          ASF subversion and git services added a comment -

          Commit 1694675 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1694675 ]

          SOLR-7849: Verify all nodes have received the new security config

          Show
          ASF subversion and git services added a comment - Commit 1694675 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1694675 ] SOLR-7849 : Verify all nodes have received the new security config
          Hide
          ASF subversion and git services added a comment -

          Commit 1694681 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1694681 ]

          SOLR-7849: avoid re-regisetring pkiAuthentication plugin http interceptor

          Show
          ASF subversion and git services added a comment - Commit 1694681 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1694681 ] SOLR-7849 : avoid re-regisetring pkiAuthentication plugin http interceptor
          Hide
          ASF subversion and git services added a comment -

          Commit 1694683 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1694683 ]

          SOLR-7849: avoid re-regisetring pkiAuthentication plugin http interceptor

          Show
          ASF subversion and git services added a comment - Commit 1694683 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1694683 ] SOLR-7849 : avoid re-regisetring pkiAuthentication plugin http interceptor
          Hide
          ASF subversion and git services added a comment -

          Commit 1694867 from Noble Paul in branch 'dev/trunk'
          [ https://svn.apache.org/r1694867 ]

          SOLR-7849: Hardening tests

          Show
          ASF subversion and git services added a comment - Commit 1694867 from Noble Paul in branch 'dev/trunk' [ https://svn.apache.org/r1694867 ] SOLR-7849 : Hardening tests
          Hide
          ASF subversion and git services added a comment -

          Commit 1694868 from Noble Paul in branch 'dev/branches/branch_5x'
          [ https://svn.apache.org/r1694868 ]

          SOLR-7849: Hardening tests

          Show
          ASF subversion and git services added a comment - Commit 1694868 from Noble Paul in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1694868 ] SOLR-7849 : Hardening tests
          Hide
          Shalin Shekhar Mangar added a comment -

          Bulk close for 5.3.0 release

          Show
          Shalin Shekhar Mangar added a comment - Bulk close for 5.3.0 release

            People

            • Assignee:
              Noble Paul
              Reporter:
              Noble Paul
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development