Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10951

Update Hadoop dependencies to 2.8.1, so Solr works with Java 9

    XMLWordPrintableJSON

    Details

    • Type: Task
    • Status: Closed
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 6.6, 7.0
    • Fix Version/s: 7.0
    • Component/s: Hadoop Integration, hdfs
    • Labels:

      Description

      See issue: HADOOP-14586

      Since Java 9 build 175 (the first Java 9 relaese candidate), Hadoop integration fails in Java 9:

         [junit4]   2> 129956 ERROR (jetty-launcher-232-thread-2) [    ] o.a.s.c.SolrCore null:java.lang.ExceptionInInitializerError
         [junit4]   2> 	at org.apache.hadoop.util.StringUtils.<clinit>(StringUtils.java:80)
         [junit4]   2> 	at org.apache.hadoop.conf.Configuration.getBoolean(Configuration.java:1437)
         [junit4]   2> 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenManager.<init>(DelegationTokenManager.java:115)
         [junit4]   2> 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.initTokenManager(DelegationTokenAuthenticationHandler.java:148)
         [junit4]   2> 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationHandler.init(DelegationTokenAuthenticationHandler.java:118)
         [junit4]   2> 	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.initializeAuthHandler(AuthenticationFilter.java:238)
         [junit4]   2> 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.initializeAuthHandler(DelegationTokenAuthenticationFilter.java:209)
         [junit4]   2> 	at org.apache.solr.security.HadoopAuthFilter.initializeAuthHandler(HadoopAuthFilter.java:120)
         [junit4]   2> 	at org.apache.hadoop.security.authentication.server.AuthenticationFilter.init(AuthenticationFilter.java:227)
         [junit4]   2> 	at org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticationFilter.init(DelegationTokenAuthenticationFilter.java:175)
         [junit4]   2> 	at org.apache.solr.security.HadoopAuthFilter.init(HadoopAuthFilter.java:68)
         [junit4]   2> 	at org.apache.solr.security.HadoopAuthPlugin.init(HadoopAuthPlugin.java:142)
         [junit4]   2> 	at org.apache.solr.core.CoreContainer.initializeAuthenticationPlugin(CoreContainer.java:360)
         [junit4]   2> 	at org.apache.solr.core.CoreContainer.reloadSecurityProperties(CoreContainer.java:684)
         [junit4]   2> 	at org.apache.solr.core.CoreContainer.load(CoreContainer.java:522)
         [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.createCoreContainer(SolrDispatchFilter.java:257)
         [junit4]   2> 	at org.apache.solr.servlet.SolrDispatchFilter.init(SolrDispatchFilter.java:177)
         [junit4]   2> 	at org.eclipse.jetty.servlet.FilterHolder.initialize(FilterHolder.java:137)
         [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:873)
         [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.updateMappings(ServletHandler.java:1565)
         [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.setFilterMappings(ServletHandler.java:1599)
         [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterMapping(ServletHandler.java:1285)
         [junit4]   2> 	at org.eclipse.jetty.servlet.ServletHandler.addFilterWithMapping(ServletHandler.java:1130)
         [junit4]   2> 	at org.eclipse.jetty.servlet.ServletContextHandler.addFilter(ServletContextHandler.java:448)
         [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner$1.lifeCycleStarted(JettySolrRunner.java:306)
         [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.setStarted(AbstractLifeCycle.java:179)
         [junit4]   2> 	at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:69)
         [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:394)
         [junit4]   2> 	at org.apache.solr.client.solrj.embedded.JettySolrRunner.start(JettySolrRunner.java:367)
         [junit4]   2> 	at org.apache.solr.cloud.MiniSolrCloudCluster.startJettySolrRunner(MiniSolrCloudCluster.java:382)
         [junit4]   2> 	at org.apache.solr.cloud.MiniSolrCloudCluster.lambda$new$0(MiniSolrCloudCluster.java:245)
         [junit4]   2> 	at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)
         [junit4]   2> 	at org.apache.solr.common.util.ExecutorUtil$MDCAwareThreadPoolExecutor.lambda$execute$0(ExecutorUtil.java:188)
         [junit4]   2> 	at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1161)
         [junit4]   2> 	at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
         [junit4]   2> 	at java.base/java.lang.Thread.run(Thread.java:844)
         [junit4]   2> Caused by: java.lang.StringIndexOutOfBoundsException: begin 0, end 3, length 1
         [junit4]   2> 	at java.base/java.lang.String.checkBoundsBeginEnd(String.java:3116)
         [junit4]   2> 	at java.base/java.lang.String.substring(String.java:1885)
         [junit4]   2> 	at org.apache.hadoop.util.Shell.<clinit>(Shell.java:51)
         [junit4]   2> 	... 36 more
         [junit4]   2> 
      

      This means we can for now only disable all tests using Hadoop, if Java 9 is detected.

      On Policeman Jenkins the HDFS integration is already disabled, but there are 2 more tests that fail:

      • org.apache.solr.security.hadoop.TestDelegationWithHadoopAuth
      • org.apache.solr.cloud.TestSolrCloudWithDelegationTokens

      I will add an assumeFalse for a while, until Hadoop fixed the issue.

        Attachments

        1. SOLR-10951.patch
          51 kB
          Uwe Schindler
        2. SOLR-10951.patch
          24 kB
          Uwe Schindler

          Issue Links

            Activity

              People

              • Assignee:
                uschindler Uwe Schindler
                Reporter:
                uschindler Uwe Schindler
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: