Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-8048

bin/solr script should accept user name and password for basicauth

    Details

      Description

      Should be able to add the line in solr.in.sh to support basic auth in the bin/solr script

      SOLR_AUTHENTICATION_OPTS="-Dbasicauth=solr:SolrRocks"
      

        Issue Links

          Activity

          Hide
          danizen Daniel Davis added a comment -

          Difficult to do well. We can respond to the authentication request sent by the server, but handling basic, MD5, etc. won't cover all the possible scenarios. I work in the federal government - we are urged not to have our applications in the line of authentication at all, but to use redirect federated authentication scenarios such as CAS, Oauth2, OpenID, etc. but usually not with all OpenID providers

          So, teaching Solr scripts such as bin/solr with the ability to interact with all such authentication plugins could not easily happen, unless the redirection somehow identifies itself as a security challenge. I'll look into that and report back.

          Show
          danizen Daniel Davis added a comment - Difficult to do well. We can respond to the authentication request sent by the server, but handling basic, MD5, etc. won't cover all the possible scenarios. I work in the federal government - we are urged not to have our applications in the line of authentication at all, but to use redirect federated authentication scenarios such as CAS, Oauth2, OpenID, etc. but usually not with all OpenID providers So, teaching Solr scripts such as bin/solr with the ability to interact with all such authentication plugins could not easily happen, unless the redirection somehow identifies itself as a security challenge. I'll look into that and report back.
          Hide
          danizen Daniel Davis added a comment -

          Since bin/solr and SolrJ would use HC UsernamePasswordCredentials, I vote for not using preemptive authentication so as to be able to support Digest authentication when done on the server side.

          Show
          danizen Daniel Davis added a comment - Since bin/solr and SolrJ would use HC UsernamePasswordCredentials , I vote for not using preemptive authentication so as to be able to support Digest authentication when done on the server side.
          Hide
          noble.paul Noble Paul added a comment -

          Daniel Davis This is designed for the basic auth authentication scheme. For basic authentication, every request must carry the credentials in the header. So, an Oauth type authentication is not possible

          Show
          noble.paul Noble Paul added a comment - Daniel Davis This is designed for the basic auth authentication scheme. For basic authentication, every request must carry the credentials in the header. So, an Oauth type authentication is not possible
          Hide
          danizen Daniel Davis added a comment -

          I dug into this a little more. The current intent seems to be that the users puts this into SOLR_INCLUDE file as

          SOLR_AUTHENTICATION_CLIENT_CONFIGURER=
          SOLR_AUTHENTICATION_OPTS=
          

          That is nice, but I agree with the intent of this bug - make it a little easier for users to figure out how to get it to work.

          I also don't quite see how this currently works. There is an HC BasicAuthenticationHttpClientConfigurer, but it requires a username and password in the constructor, and SolrCLI doesn't seem to provide one, so I'm not sure how it can work.

          Show
          danizen Daniel Davis added a comment - I dug into this a little more. The current intent seems to be that the users puts this into SOLR_INCLUDE file as SOLR_AUTHENTICATION_CLIENT_CONFIGURER= SOLR_AUTHENTICATION_OPTS= That is nice, but I agree with the intent of this bug - make it a little easier for users to figure out how to get it to work. I also don't quite see how this currently works. There is an HC BasicAuthenticationHttpClientConfigurer , but it requires a username and password in the constructor, and SolrCLI doesn't seem to provide one, so I'm not sure how it can work.
          Hide
          danizen Daniel Davis added a comment -

          This is designed for the basic auth authentication scheme. For basic authentication, every request must carry the credentials in the header.

          Acknowledged. Its also supposed to be over SSL, so its not too bad.

          Show
          danizen Daniel Davis added a comment - This is designed for the basic auth authentication scheme. For basic authentication, every request must carry the credentials in the header. Acknowledged. Its also supposed to be over SSL, so its not too bad.
          Hide
          danizen Daniel Davis added a comment -

          Noble Paul, I'd like to take this bug, but I'm not a committer. Still, I see directly where in SolrCLI this would be added to HttpClient. If you have a patch already, no reason to wait for me. I will check this bug before uploading a patch.

          Show
          danizen Daniel Davis added a comment - Noble Paul , I'd like to take this bug, but I'm not a committer. Still, I see directly where in SolrCLI this would be added to HttpClient. If you have a patch already, no reason to wait for me. I will check this bug before uploading a patch.
          Hide
          noble.paul Noble Paul added a comment -

          A ticket is a bug when it works wrongly. SolrJ works as advertised. BasicAuth is a new feature in the server and we are adding the support to client as well

          However , please feel free to submit a patch

          Show
          noble.paul Noble Paul added a comment - A ticket is a bug when it works wrongly. SolrJ works as advertised. BasicAuth is a new feature in the server and we are adding the support to client as well However , please feel free to submit a patch
          Hide
          danizen Daniel Davis added a comment -

          True enough.

          Show
          danizen Daniel Davis added a comment - True enough.
          Hide
          janhoy Jan Høydahl added a comment -

          Anyone have a patch already? Daniel Davis?

          Show
          janhoy Jan Høydahl added a comment - Anyone have a patch already? Daniel Davis ?
          Hide
          risdenk Kevin Risden added a comment -

          Since SOLR-8053 was merged, this should be easier and not require the HttpClient modifications.

          Show
          risdenk Kevin Risden added a comment - Since SOLR-8053 was merged, this should be easier and not require the HttpClient modifications.
          Hide
          harcor harcor added a comment -

          Could it be implemented in a such way that the solr script would take a path to obfuscated user/password file instead of command line user/password?

          Show
          harcor harcor added a comment - Could it be implemented in a such way that the solr script would take a path to obfuscated user/password file instead of command line user/password?
          Hide
          danizen Daniel Davis added a comment -

          This is a good idea - passwords on the command-line are frowned upon.

          Show
          danizen Daniel Davis added a comment - This is a good idea - passwords on the command-line are frowned upon.
          Hide
          danizen Daniel Davis added a comment -

          I will work on this tomorrow.

          Show
          danizen Daniel Davis added a comment - I will work on this tomorrow.
          Hide
          janhoy Jan Høydahl added a comment -

          The description suggests:

          ...or alternately it should prompt for user name and password

          That is an approach taken by many other tools, and avoids getting PWs in the shell history etc. However for scripting, a separate pw file is more secure.

          Show
          janhoy Jan Høydahl added a comment - The description suggests: ...or alternately it should prompt for user name and password That is an approach taken by many other tools, and avoids getting PWs in the shell history etc. However for scripting, a separate pw file is more secure.
          Hide
          danizen Daniel Davis added a comment -

          So, in looking at SolrCLI, I don't see how SOLR-8053 helps much. SolrCLI usually does not create requests but rather goes directly through the JSON API using routines such as getJson() and postJsonToSolr(). As I thought before, the right place to add basic auth seems to be in getHttpClient().

          However, I do also see some places where for cloud tools, a CloudSolrClient is created. There, something else is needed.

          Finally, I had somehow expected that getCommonToolOptions() would do just that, and I would be able there to add options. I see now that would be a mistake - the ExampleTool does not need an authentication option, because it will not be creating a cloud that uses security.json.

          For now, I will make a start by making sure that CreateTool() works properly, with a manually added Option for that tool. I will therefore also be handling CreateCollectionTool(). Once that works, I'll have to slowly work forward to other tools, and I will need help to know that I've got all of them that need this option.

          How the username and password are passed in is less important to me right now - I can iterate on that once I have a basic mechanism.

          Show
          danizen Daniel Davis added a comment - So, in looking at SolrCLI, I don't see how SOLR-8053 helps much. SolrCLI usually does not create requests but rather goes directly through the JSON API using routines such as getJson() and postJsonToSolr() . As I thought before, the right place to add basic auth seems to be in getHttpClient() . However, I do also see some places where for cloud tools, a CloudSolrClient is created. There, something else is needed. Finally, I had somehow expected that getCommonToolOptions() would do just that, and I would be able there to add options. I see now that would be a mistake - the ExampleTool does not need an authentication option, because it will not be creating a cloud that uses security.json. For now, I will make a start by making sure that CreateTool() works properly, with a manually added Option for that tool. I will therefore also be handling CreateCollectionTool() . Once that works, I'll have to slowly work forward to other tools, and I will need help to know that I've got all of them that need this option. How the username and password are passed in is less important to me right now - I can iterate on that once I have a basic mechanism.
          Hide
          danizen Daniel Davis added a comment -

          Noble Paul, interesting that TestAuthenticationFramework uses an HttpClientConfigurator. I will try to use that mechanism, and see how far I get. One hesitation I have is that others may use it for different things - this is something I rely on committers to either know or check on... it is certainly safer to use something in a test case than in SolrCLI.

          Show
          danizen Daniel Davis added a comment - Noble Paul , interesting that TestAuthenticationFramework uses an HttpClientConfigurator . I will try to use that mechanism, and see how far I get. One hesitation I have is that others may use it for different things - this is something I rely on committers to either know or check on... it is certainly safer to use something in a test case than in SolrCLI.
          Hide
          noble.paul Noble Paul added a comment -

          Daniel Davis The mechanism used in TestAuthenticationFramework is suboptimal. But it would work because the solrcli is a single use application.

          Show
          noble.paul Noble Paul added a comment - Daniel Davis The mechanism used in TestAuthenticationFramework is suboptimal. But it would work because the solrcli is a single use application.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 62452f033a3945d2812fa17ab07cfbe7248bb439 in lucene-solr's branch refs/heads/branch_6x from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=62452f0 ]

          SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh

          Show
          jira-bot ASF subversion and git services added a comment - Commit 62452f033a3945d2812fa17ab07cfbe7248bb439 in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=62452f0 ] SOLR-8048 : bin/solr script should support basic auth credentials provided in solr.in.sh
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 97e696dd506aa01142c8456452c6f66451dd5430 in lucene-solr's branch refs/heads/apiv2 from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=97e696d ]

          SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh

          Show
          jira-bot ASF subversion and git services added a comment - Commit 97e696dd506aa01142c8456452c6f66451dd5430 in lucene-solr's branch refs/heads/apiv2 from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=97e696d ] SOLR-8048 : bin/solr script should support basic auth credentials provided in solr.in.sh
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 2e101c42ca6c6a4e03cf3a1ab1010f5995dccd88 in lucene-solr's branch refs/heads/apiv2 from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2e101c4 ]

          SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh

          Show
          jira-bot ASF subversion and git services added a comment - Commit 2e101c42ca6c6a4e03cf3a1ab1010f5995dccd88 in lucene-solr's branch refs/heads/apiv2 from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2e101c4 ] SOLR-8048 : bin/solr script should support basic auth credentials provided in solr.in.sh
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 5ee4e8a6141b6d9ac0016e82b6561bca9587faf0 in lucene-solr's branch refs/heads/apiv2 from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5ee4e8a ]

          SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh

          Show
          jira-bot ASF subversion and git services added a comment - Commit 5ee4e8a6141b6d9ac0016e82b6561bca9587faf0 in lucene-solr's branch refs/heads/apiv2 from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5ee4e8a ] SOLR-8048 : bin/solr script should support basic auth credentials provided in solr.in.sh
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 5eabffc79754f533654bcbc73ab6441e6059d45f in lucene-solr's branch refs/heads/apiv2 from Noble Paul
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5eabffc ]

          SOLR-8048: bin/solr script should support basic auth credentials provided in solr.in.sh

          Show
          jira-bot ASF subversion and git services added a comment - Commit 5eabffc79754f533654bcbc73ab6441e6059d45f in lucene-solr's branch refs/heads/apiv2 from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5eabffc ] SOLR-8048 : bin/solr script should support basic auth credentials provided in solr.in.sh
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 2d6b7ea966774af39fb131c09835768f33958d05 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2d6b7ea ]

          SOLR-8048: Close the http client in a finally clause at the end of the test

          Show
          jira-bot ASF subversion and git services added a comment - Commit 2d6b7ea966774af39fb131c09835768f33958d05 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2d6b7ea ] SOLR-8048 : Close the http client in a finally clause at the end of the test
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 8653be9a5bb0eaa22d96fddf09dd507ad7a94cd4 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8653be9 ]

          SOLR-8048: Stop using deprecated CollectionAdminRequest.Reload constructor

          Show
          jira-bot ASF subversion and git services added a comment - Commit 8653be9a5bb0eaa22d96fddf09dd507ad7a94cd4 in lucene-solr's branch refs/heads/master from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=8653be9 ] SOLR-8048 : Stop using deprecated CollectionAdminRequest.Reload constructor
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 31316f7682f47acdbbd3367e9067b093f25044e5 in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=31316f7 ]

          SOLR-8048: Close the http client in a finally clause at the end of the test
          (cherry picked from commit 2d6b7ea)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 31316f7682f47acdbbd3367e9067b093f25044e5 in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=31316f7 ] SOLR-8048 : Close the http client in a finally clause at the end of the test (cherry picked from commit 2d6b7ea)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 119ea15ad6efaaf2c3b2d833820f152998599b2e in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=119ea15 ]

          SOLR-8048: Stop using deprecated CollectionAdminRequest.Reload constructor
          (cherry picked from commit 8653be9)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 119ea15ad6efaaf2c3b2d833820f152998599b2e in lucene-solr's branch refs/heads/branch_6x from Shalin Shekhar Mangar [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=119ea15 ] SOLR-8048 : Stop using deprecated CollectionAdminRequest.Reload constructor (cherry picked from commit 8653be9)
          Hide
          shalinmangar Shalin Shekhar Mangar added a comment -

          I saw some jenkins failure where this test was leaking http client so I pushed a fix to always close the client in a finally clause.

          Show
          shalinmangar Shalin Shekhar Mangar added a comment - I saw some jenkins failure where this test was leaking http client so I pushed a fix to always close the client in a finally clause.
          Hide
          mikemccand Michael McCandless added a comment -

          Bulk close resolved issues after 6.2.0 release.

          Show
          mikemccand Michael McCandless added a comment - Bulk close resolved issues after 6.2.0 release.

            People

            • Assignee:
              noble.paul Noble Paul
              Reporter:
              noble.paul Noble Paul
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development