Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9861

Support Cross Data Center Replication under HDFS

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 6.3
    • Fix Version/s: None
    • Component/s: CDCR, Hadoop Integration, hdfs
    • Labels:
      None

      Description

      Currently Solr Cloud does not support CDCR under HDFS (according to http://search-lucene.com/m/Solr/eHNlLSriaVCpuU1?subj=Re+Using+Solr+CDCR+with+HdfsDirectoryFactory).
      If the CDCR configuration is performed according to

      https://cwiki.apache.org/confluence/pages/viewpage.action?pageId=62687462

      and at the same time the HdfsDirectoryFactory is configured on both Source cluster and Target cluster according to

      https://cwiki.apache.org/confluence/display/solr/Running+Solr+on+HDFS

      it leads to the following error message during reload of the collection “collection_source” of the source cluster:

      2016-11-23 12:05:35.604 ERROR (qtp1134712904-8045) [c:collection_source s:shard1 r:core_node1 x:collection_source_shard1_replica1] o.a.s.s.HttpSolrCall null:org.apache.solr.common.SolrException: Error handling 'reload' action
              at org.apache.solr.handler.admin.CoreAdminOperation$3.call(CoreAdminOperation.java:150)
              at org.apache.solr.handler.admin.CoreAdminHandler$CallInfo.call(CoreAdminHandler.java:367)
              at org.apache.solr.handler.admin.CoreAdminHandler.handleRequestBody(CoreAdminHandler.java:158)
              at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:156)
              at org.apache.solr.servlet.HttpSolrCall.handleAdminRequest(HttpSolrCall.java:663)
              at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:445)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:257)
              at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:208)
              at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
              at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
              at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
              at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
              at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
              at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
              at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
              at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
              at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
              at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
              at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
              at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
              at org.eclipse.jetty.server.Server.handle(Server.java:518)
              at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
              at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
              at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
              at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
              at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
              at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
              at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
              at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
              at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
              at java.lang.Thread.run(Thread.java:745)
      Caused by: org.apache.solr.common.SolrException: Unable to reload core [collection_source_shard1_replica1]
              at org.apache.solr.core.CoreContainer.reload(CoreContainer.java:920)
              at org.apache.solr.handler.admin.CoreAdminOperation$3.call(CoreAdminOperation.java:148)
              ... 31 more
      Caused by: org.apache.solr.common.SolrException: Solr instance is not configured with the cdcr update log.
      

      However, the solr instance was configured correctly. CDCR should work in the same way with HdfsDirectoryFactory as it works with StandardDirectoryFactory or NRTCachingDirectoryFactory.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              solreclipse Gero Willmes
            • Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated: