Solr
  1. Solr
  2. SOLR-2115

DataImportHandler config file *must* be specified in "defaults" or status will be "DataImportHandler started. Not Initialized. No commands can be run"

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 1.4.1, 1.4.2, 3.1, 4.0-ALPHA
    • Fix Version/s: 4.0-BETA
    • Labels:
      None

      Description

      The DataImportHandler has two URL parameters for defining the data-config.xml file to be used for the command. 'config' is used in some places and 'dataConfig' is used in other places.

      'config' does not work from an HTTP request. However, if it is in the "defaults" section of the DIH <requestHandler> definition, it works. If the 'config' parameter is used in an HTTP request, the DIH uses the default in the <requestHandler> anyway.

      This is the exception stack recieved by the client if there is no default. (This is the 3.X branch.)

      <html>
      <head>
      <meta http-equiv="Content-Type" content="text/html; charset=ISO-8859-1"/>
      <title>Error 500 </title>
      </head>
      <body><h2>HTTP ERROR: 500</h2><pre>null

      java.lang.NullPointerException
      at org.apache.solr.handler.dataimport.DataImportHandler.handleRequestBody(DataImportHandler.java:146)
      at org.apache.solr.handler.RequestHandlerBase.handleRequest(RequestHandlerBase.java:131)
      at org.apache.solr.core.SolrCore.execute(SolrCore.java:1316)
      at org.apache.solr.servlet.SolrDispatchFilter.execute(SolrDispatchFilter.java:338)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:241)
      at org.mortbay.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1089)
      ..etc..

      1. SOLR-2115.patch
        22 kB
        James Dyer
      2. SOLR-2115.patch
        22 kB
        James Dyer

        Issue Links

          Activity

          Hide
          Hoss Man added a comment -

          Updating summary to help people identify this issue when trouble shooting

          if you do not have a "defaults" section when DataImportHandler is declared, then the DataImporter will never be initialized, and the "status" returned by the handler will be...

          DataImportHandler started. Not Initialized. No commands can be run

          ...any time you try to do an import.

          Skmming the code, it seems that there may be some situations in which you can force it to reload the config, using a file name specified using a request param (see lance's earlier comment about hte confusion of what that param is)

          Show
          Hoss Man added a comment - Updating summary to help people identify this issue when trouble shooting if you do not have a "defaults" section when DataImportHandler is declared, then the DataImporter will never be initialized, and the "status" returned by the handler will be... DataImportHandler started. Not Initialized. No commands can be run ...any time you try to do an import. Skmming the code, it seems that there may be some situations in which you can force it to reload the config, using a file name specified using a request param (see lance's earlier comment about hte confusion of what that param is)
          Hide
          Robert Muir added a comment -
          Show
          Robert Muir added a comment - Moving out all unassigned issues set to 3.1 per this email: http://www.lucidimagination.com/search/document/f026cc56081b5a51/unassigned_jira_issues_set_for_3_1
          Hide
          Robert Muir added a comment -

          Bulk move 3.2 -> 3.3

          Show
          Robert Muir added a comment - Bulk move 3.2 -> 3.3
          Hide
          Robert Muir added a comment -

          3.4 -> 3.5

          Show
          Robert Muir added a comment - 3.4 -> 3.5
          Hide
          Hoss Man added a comment -

          Bulk changing fixVersion 3.6 to 4.0 for any open issues that are unassigned and have not been updated since March 19.

          Email spam suppressed for this bulk edit; search for hoss20120323nofix36 to identify all issues edited

          Show
          Hoss Man added a comment - Bulk changing fixVersion 3.6 to 4.0 for any open issues that are unassigned and have not been updated since March 19. Email spam suppressed for this bulk edit; search for hoss20120323nofix36 to identify all issues edited
          Hide
          Hoss Man added a comment -

          bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment

          Show
          Hoss Man added a comment - bulk fixing the version info for 4.0-ALPHA and 4.0 all affected issues have "hoss20120711-bulk-40-change" in comment
          Hide
          James Dyer added a comment -

          With this patch...

          • DIH attempts to reload the configuration every time a new import is started. This is slightly more overhead, but negligible compared with the time an import takes as a whole.
          • The config is not loaded on startup and there is no need to have a <defaults /> section or have the config declared in solrconfig.xml at all. Instead users have the option to specify the config file on the request with the "config" parameter.
          • The "dataConfig" parameter, which lets users include the entire configuration as a request parameter is now always supported (previously this was only supported in debug mode)

          -The "reload-config" command is still supported, which is useful for validating a new configuration file, or if you want to specify a file, load it, and not have it reloaded again on import.

          • Datasources can still be specified in solrconfig.xml. As before these must be specified in the "defaults" section of the handler in solrconfig.xml. However, these are not parsed until the main configuration is loaded.
          • If there is an xml mistake in the configuration a much more user-friendly message is given in xml format, not raw format as before. Users can fix the problem and "reload-config".
          Show
          James Dyer added a comment - With this patch... DIH attempts to reload the configuration every time a new import is started. This is slightly more overhead, but negligible compared with the time an import takes as a whole. The config is not loaded on startup and there is no need to have a <defaults /> section or have the config declared in solrconfig.xml at all. Instead users have the option to specify the config file on the request with the "config" parameter. The "dataConfig" parameter, which lets users include the entire configuration as a request parameter is now always supported (previously this was only supported in debug mode) -The "reload-config" command is still supported, which is useful for validating a new configuration file, or if you want to specify a file, load it, and not have it reloaded again on import. Datasources can still be specified in solrconfig.xml. As before these must be specified in the "defaults" section of the handler in solrconfig.xml. However, these are not parsed until the main configuration is loaded. If there is an xml mistake in the configuration a much more user-friendly message is given in xml format, not raw format as before. Users can fix the problem and "reload-config".
          Hide
          Alexey Serba added a comment -

          Very cool! It would be great to have single dataimport handler that serves multiple imports at the same time...

          Show
          Alexey Serba added a comment - Very cool! It would be great to have single dataimport handler that serves multiple imports at the same time...
          Hide
          James Dyer added a comment -

          yeah, I've wanted that too, and this is a step in that direction.

          Show
          James Dyer added a comment - yeah, I've wanted that too, and this is a step in that direction.
          Hide
          James Dyer added a comment -

          Updated patch, which I plan to commit soon.

          Show
          James Dyer added a comment - Updated patch, which I plan to commit soon.
          Hide
          James Dyer added a comment -

          Committed. Trunk: r1368993, branch_4x (beta): r1369009

          Show
          James Dyer added a comment - Committed. Trunk: r1368993, branch_4x (beta): r1369009

            People

            • Assignee:
              James Dyer
              Reporter:
              Lance Norskog
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development