Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-12283

Unable To Load ZKPropertiesWriter when dih.jar is added as runtimelib BLOB in .system collection

    XMLWordPrintableJSON

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Major
    • Resolution: Won't Fix
    • Affects Version/s: 6.6.1, 7.3
    • Fix Version/s: None
    • Labels:
      None
    • Environment:

      Debian
      SolrCloud

      Description

      Hello,

      It's been 2 weeks that I try to correct this problem with the community user-solr but no success. I seriously wonder if this is not a problem in the code. I do not have the impression that many people use DIH with Solr's cloud version.
      On Internet, no similar problem.

      For information, the following configuration of DIH comes from DIHs that work in production on a single Solr server. The connections to the databases are therefore correct.

      Errors messages:

      DataImporter
      Full Import failed:org.apache.solr.handler.dataimport.DataImportHandlerException: Unable to PropertyWriter implementation:ZKPropertiesWriter
      	at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:339)
      	at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:420)
      	at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
      	at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:183)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
      	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
      	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.Server.handle(Server.java:530)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
      	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
      	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
      	at java.lang.Thread.run(Thread.java:748)
      Caused by: java.lang.NullPointerException
      	at org.apache.solr.handler.dataimport.DocBuilder.loadClass(DocBuilder.java:935)
      	at org.apache.solr.handler.dataimport.DataImporter.createPropertyWriter(DataImporter.java:335)
      	... 42 more
      
      RequestHandlerBase
      java.lang.NullPointerException
      	at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:429)
      	at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
      	at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:183)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
      	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
      	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.Server.handle(Server.java:530)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
      	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
      	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
      	at java.lang.Thread.run(Thread.java:748)
      
      HttpSolrCall
      null:java.lang.NullPointerException
      	at org.apache.solr.handler.dataimport.DataImporter.doFullImport(DataImporter.java:429)
      	at org.apache.solr.handler.dataimport.DataImporter.runCmd(DataImporter.java:483)
      	at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:183)
      	at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:195)
      	at org.apache.solr.core.SolrCore.execute(SolrCore.java:2503)
      	at org.apache.solr.servlet.HttpSolrCall.execute(HttpSolrCall.java:711)
      	at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:517)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:384)
      	at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:330)
      	at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1629)
      	at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:533)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      	at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:190)
      	at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:1595)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextHandle(ScopedHandler.java:188)
      	at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1253)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:168)
      	at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:473)
      	at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:1564)
      	at org.eclipse.jetty.server.handler.ScopedHandler.nextScope(ScopedHandler.java:166)
      	at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1155)
      	at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      	at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:219)
      	at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:126)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
      	at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:132)
      	at org.eclipse.jetty.server.Server.handle(Server.java:530)
      	at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:347)
      	at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:256)
      	at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:279)
      	at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:102)
      	at org.eclipse.jetty.io.ChannelEndPoint$2.run(ChannelEndPoint.java:124)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.doProduce(EatWhatYouKill.java:247)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.produce(EatWhatYouKill.java:140)
      	at org.eclipse.jetty.util.thread.strategy.EatWhatYouKill.run(EatWhatYouKill.java:131)
      	at org.eclipse.jetty.util.thread.ReservedThreadExecutor$ReservedThread.run(ReservedThreadExecutor.java:382)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:708)
      	at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:626)
      	at java.lang.Thread.run(Thread.java:748)
      

      Process to reproduce the problem:

      Create .system collection for the BLOB

      curl http://localhost:8983/solr/admin/collections?action=CREATE&name=.system
      

      Add Data Import Handler JAR file.

      curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @solr-dataimporthandler-7.3.0.jar http://localhost:8983/solr/.system/blob/data-import-handler
      

      Add the runtime lib on the configoverlay.json with the API

      curl http://localhost:8983/solr/unicorn/config -H 'Content-type:application/json' -d '{"add-runtimelib": { "name":"data-import-handler", "version":1 }}'
      

      Send mysql connector on the blob. I have test with the connector-bin.jar but I have the same error.

      curl -X POST -H 'Content-Type: application/octet-stream' --data-binary @mysql-connector-java-5.1.46.jar http://localhost:8983/solr/.system/blob/mysql-connector-java
      

      Add runtime

      curl http://localhost:8983/solr/unicorn/config -H 'Content-type:application/json' -d '{"add-runtimelib": { "name":"mysql-connector-java", "version":1 }}'
      

      Personally, I create a DIH folder on the collection config path on the cloud. It's not necessary.

      And I put the DIH config file on the cloud with the zkcli (it's a personnal script to overload the zkcli.sh on Solr but it's the same result. My file is on the cloud.)

      ./zkcli.py putfile /configs/unicorn/DIH/ DIH/indexation_events.xml
      

      Add request handler config:

      curl -X POST http://{solr_server}:8983/solr/{collection_name}/config -H 'Content-type:application/json' -d '
      {
        "add-requesthandler":
        {
          "name": "/test-dih",
          "class": "org.apache.solr.handler.dataimport.DataImportHandler",
          "runtimeLib": true,
          "version": 1,
          "defaults":
          {
            "config": "DIH/indexation_events.xml"
          }
        }
      } '
      

      Reload the config:

      curl http://localhost:8983/solr/unicorn/test-dih?command=reload-config&commit=true&debug=true
      

      Import and have error:

      curl http://localhost:8983/solr/unicorn/test-dih?command=full-import&commit=true&debug=true
      

      I have test to add on request handler and BLOB this jar files but not change:

      1. data-import-handler-extras
      2. solr-core
      3. solr-solrj

      I have join all files used in this process.

      Thansk you.

        Attachments

        1. request_handler_config.json
          0.2 kB
          Maxence SAUNIER
        2. mysql-connector-java-5.1.46-bin.jar
          981 kB
          Maxence SAUNIER
        3. mysql-connector-java-5.1.46.jar
          981 kB
          Maxence SAUNIER
        4. solr-dataimporthandler-extras-7.3.0.jar
          39 kB
          Maxence SAUNIER
        5. solr-dataimporthandler-7.3.0.jar
          223 kB
          Maxence SAUNIER
        6. solr-solrj-7.3.0.jar
          1.66 MB
          Maxence SAUNIER
        7. solr-core-7.3.0.jar
          4.96 MB
          Maxence SAUNIER
        8. solr.log
          6.68 MB
          Maxence SAUNIER
        9. modified-DIH.zip
          218 kB
          Shawn Heisey
        10. solr.log
          7.11 MB
          Maxence SAUNIER
        11. solr.log
          78 kB
          Maxence SAUNIER
        12. modified-DIH.zip
          218 kB
          Shawn Heisey
        13. solr.log
          89 kB
          Maxence SAUNIER

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                Moltroon Maxence SAUNIER
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: