Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9542

Kerberos delegation tokens requires missing Jackson library

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      GET, RENEW or CANCEL operations for the delegation tokens support requires the Solr server to have old jackson added as a dependency.

      Steps to reproduce the problem:
      1) Configure Solr to use delegation tokens
      2) Start Solr
      3) Use a SolrJ application to get a delegation token.

      The server throws the following:

      java.lang.NoClassDefFoundError: org/codehaus/jackson/map/ObjectMapper
              at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.managementOperation(DelegationTokenAuthenticationHandler.java:279)
              at org.apache.solr.security.KerberosPlugin$RequestContinuesRecorderAuthenticationHandler.managementOperation(KerberosPlugin.java:566)
              at org.apache.hadoop.security.authentication.server.AuthenticationFilter.doFilter(AuthenticationFilter.java:514)
              at org.apache.solr.security.DelegationTokenKerberosFilter.doFilter(DelegationTokenKerberosFilter.java:123)
              at org.apache.solr.security.KerberosPlugin.doAuthenticate(KerberosPlugin.java:265)
              at org.apache.solr.servlet.SolrDispatchFilter.authenticateRequest(SolrDispatchFilter.java:318)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:222)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
              at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
              at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
              at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
              at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
              at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
              at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
              at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
              at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
              at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
              at org.eclipse.jetty.server.Server.handle(Server.java:518)
              at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
              at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
              at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
              at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
              at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
              at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
              at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
              at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
              at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
              at java.lang.Thread.run(Thread.java:745)
      
      1. SOLR-9542.patch
        1 kB
        Ishan Chattopadhyaya

        Activity

        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        I've added HADOOP-13672 for hadoop-auth to enable us to remove this unnecessary dependency.

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - I've added HADOOP-13672 for hadoop-auth to enable us to remove this unnecessary dependency.
        Hide
        dsmiley David Smiley added a comment -

        I don't know how deeply Jackson is required for this capability; let's say hypothetically it is and it'd be hard to switch out. If that's the case, we could simply mark this dependency as "optional" in the Maven POM, and we can add docs to the ref guide on the dependencies needed. I suspect very few people are using Kerberos to secure Solr. People care about security but use other means.

        If it's not particularly hard to switch then lets do our collective users a favor and switch to our existing JSON parsing dependency: noggit.

        Show
        dsmiley David Smiley added a comment - I don't know how deeply Jackson is required for this capability; let's say hypothetically it is and it'd be hard to switch out. If that's the case, we could simply mark this dependency as "optional" in the Maven POM, and we can add docs to the ref guide on the dependencies needed. I suspect very few people are using Kerberos to secure Solr. People care about security but use other means. If it's not particularly hard to switch then lets do our collective users a favor and switch to our existing JSON parsing dependency: noggit.
        Hide
        Timothy055 Timothy M. Rodriguez added a comment -

        Not sure it makes sense to introduce a Jackson dependency here. I'm conflicted on how big of an issue this is though. It's a really old version of jackson since it depends on the org.codehaus version. On the other hand, it's probably less likely to conflict as such.

        Show
        Timothy055 Timothy M. Rodriguez added a comment - Not sure it makes sense to introduce a Jackson dependency here. I'm conflicted on how big of an issue this is though. It's a really old version of jackson since it depends on the org.codehaus version. On the other hand, it's probably less likely to conflict as such.
        Hide
        shinichiro abe Shinichiro Abe added a comment - - edited

        Hrishikesh Gadre, I understood, thanks.
        IIUC, in SolrJ jackson library is used for DelegationTokenResponse to use ObjectMapper.
        It would be nice if we could replace jackson with noggit, for instance Utils.fromJSON(InputStream is).
        It's ok Solrj testing may depend to jackson or guava, but Solrj itself should not do unless using smile(Btw jackson-dataformat-smile is missing in Solrj deps), IMO.

        Show
        shinichiro abe Shinichiro Abe added a comment - - edited Hrishikesh Gadre , I understood, thanks. IIUC, in SolrJ jackson library is used for DelegationTokenResponse to use ObjectMapper. It would be nice if we could replace jackson with noggit, for instance Utils.fromJSON(InputStream is). It's ok Solrj testing may depend to jackson or guava, but Solrj itself should not do unless using smile(Btw jackson-dataformat-smile is missing in Solrj deps), IMO.
        Show
        hgadre Hrishikesh Gadre added a comment - Shinichiro Abe BTW solrj does not depend upon the older version of jackson library. https://github.com/apache/lucene-solr/blob/bede7aefa3b2294e869d7fa543417e160e3518f9/solr/solrj/ivy.xml#L44-L47 https://github.com/apache/lucene-solr/blob/bede7aefa3b2294e869d7fa543417e160e3518f9/solr/core/ivy.xml#L96-L97
        Hide
        hgadre Hrishikesh Gadre added a comment -

        Shinichiro Abe

        Currently jackson and guava are SolrJ dependencies for that plugin. guava is used for only one annotation, it is a large jar and usually is suppose to provided from client program. If that plugin does not have a strong dependency, would you like to make those scope provided?

        I think that guava dependency can be avoided by commenting out the VisibleForTesting annotation (since the code comment serves the same purpose as the annotation).

        Show
        hgadre Hrishikesh Gadre added a comment - Shinichiro Abe Currently jackson and guava are SolrJ dependencies for that plugin. guava is used for only one annotation, it is a large jar and usually is suppose to provided from client program. If that plugin does not have a strong dependency, would you like to make those scope provided? I think that guava dependency can be avoided by commenting out the VisibleForTesting annotation (since the code comment serves the same purpose as the annotation).
        Hide
        shinichiro abe Shinichiro Abe added a comment -

        Adding jackson just for kerberosPlugin feels like an overkill.

        So does SolrJ, I think. Currently jackson and guava are SolrJ dependencies for that plugin. guava is used for only one annotation, it is a large jar and usually is suppose to provided from client program. If that plugin does not have a strong dependency, would you like to make those scope provided? ref CONNECTORS-1338.

        Show
        shinichiro abe Shinichiro Abe added a comment - Adding jackson just for kerberosPlugin feels like an overkill. So does SolrJ, I think. Currently jackson and guava are SolrJ dependencies for that plugin. guava is used for only one annotation, it is a large jar and usually is suppose to provided from client program. If that plugin does not have a strong dependency, would you like to make those scope provided? ref CONNECTORS-1338 .
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 5acbcac274dd3f2096a3a91ee1afd2a1f03f5ed6 in lucene-solr's branch refs/heads/master from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5acbcac ]

        SOLR-9542: Kerberos delegation tokens requires Jackson library

        Show
        jira-bot ASF subversion and git services added a comment - Commit 5acbcac274dd3f2096a3a91ee1afd2a1f03f5ed6 in lucene-solr's branch refs/heads/master from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5acbcac ] SOLR-9542 : Kerberos delegation tokens requires Jackson library
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit ec5a53d706173046f2e0048abe2d6376a7e1a375 in lucene-solr's branch refs/heads/branch_6x from Noble Paul
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ec5a53d ]

        SOLR-9542: Kerberos delegation tokens requires Jackson library

        Show
        jira-bot ASF subversion and git services added a comment - Commit ec5a53d706173046f2e0048abe2d6376a7e1a375 in lucene-solr's branch refs/heads/branch_6x from Noble Paul [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=ec5a53d ] SOLR-9542 : Kerberos delegation tokens requires Jackson library
        Hide
        hgadre Hrishikesh Gadre added a comment -

        Ishan Chattopadhyaya I reviewed the patch and it looks good.

        HADOOP-13332 is tracking the work required for upgrading jackson library in Hadoop. Since the work is underway for Hadoop 3 release, this may be addressed in next few months. (BTW SOLR-9515 is tracking the work required in Solr to support Hadoop 3). But in my opinion we shouldn't hold off for this Hadoop enhancement. Instead we should commit this patch to fix the reported issue. May be we can file another JIRA to revert this change once the Hadoop side fix is available.

        Show
        hgadre Hrishikesh Gadre added a comment - Ishan Chattopadhyaya I reviewed the patch and it looks good. HADOOP-13332 is tracking the work required for upgrading jackson library in Hadoop. Since the work is underway for Hadoop 3 release, this may be addressed in next few months. (BTW SOLR-9515 is tracking the work required in Solr to support Hadoop 3). But in my opinion we shouldn't hold off for this Hadoop enhancement. Instead we should commit this patch to fix the reported issue. May be we can file another JIRA to revert this change once the Hadoop side fix is available.
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        Noble, indeed lame that we have to add the old jackson library as a dependency, just because hadoop is stuck with an old version. Btw, I think we already have jackson (from org.fasterxml.*) in core.

        Noble, Do you suggest we instruct users to download the jars themselves and add somehow them to the solr/solr.in.sh script for startup? Btw, not sure if upgrading Hadoop to use the latest jackson packages is an immediate option; I think not. Gregory Chanan, any thoughts? I am even fine adding it to solr-core; the overhead of adding this is around 750kb.

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Noble, indeed lame that we have to add the old jackson library as a dependency, just because hadoop is stuck with an old version. Btw, I think we already have jackson (from org.fasterxml.*) in core. Noble, Do you suggest we instruct users to download the jars themselves and add somehow them to the solr/solr.in.sh script for startup? Btw, not sure if upgrading Hadoop to use the latest jackson packages is an immediate option; I think not. Gregory Chanan , any thoughts? I am even fine adding it to solr-core; the overhead of adding this is around 750kb.
        Hide
        noble.paul Noble Paul added a comment -

        Adding jackson just for kerberosPlugin feels like an overkill. Can this be optional?

        Show
        noble.paul Noble Paul added a comment - Adding jackson just for kerberosPlugin feels like an overkill. Can this be optional?
        Hide
        ichattopadhyaya Ishan Chattopadhyaya added a comment -

        Here's a patch that adds the dependencies. Gregory Chanan, can you please review?

        Show
        ichattopadhyaya Ishan Chattopadhyaya added a comment - Here's a patch that adds the dependencies. Gregory Chanan , can you please review?

          People

          • Assignee:
            Unassigned
            Reporter:
            ichattopadhyaya Ishan Chattopadhyaya
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:

              Development