Solr
  1. Solr
  2. SOLR-3668

New Admin : DataImport : Specifying Custom Parameters

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.0-ALPHA
    • Fix Version/s: None
    • Component/s: web gui
    • Labels:
      None
    • Environment:

      MacOS X 10.7.4, Safari 5.1.7

      Description

      I'm trying to run the following direct call via the WebGUI:
      http://localhost:9090/solr/issues/dataimport?command=full-import&importfile=/absolute/path/to/file.xml

      The above direct call produces this log output:
      24.07.2012 15:18:40 org.apache.solr.handler.dataimport.XPathEntityProcessor initQuery
      WARNUNG: Failed for url : /absolute/path/to/file.xml

      When giving an existing file, DIH works. But this is enough to show the difference between direct call and call via WebGUI.

      Steps I do in the WebGUI:

      0. In a multicore environment where one core is called "issues"

      1. Open the tab of core "issues", and there the sub-item "Dataimport":
      http://localhost:9090/solr/#/issues/dataimport//dataimport

      2. Specify a custom parameter in the text field labled "Custom Parameters" like "importfile=/absolute/path/to/file.xml"

      Resulting log output:
      24.07.2012 15:22:47 org.apache.solr.handler.dataimport.XPathEntityProcessor initQuery
      WARNUNG: Failed for url :
      java.lang.RuntimeException: java.io.FileNotFoundException: Could not find file:

      (no filename specified)

      When trying with an existing file, the same output (no filename) is logged.

      I've tried to find out how to specify the custom parameters by looking into dataimport.js but it did not help me (I did not dwell on it, though). If it would work by specifying the parameter in a different way it would be great if a little info would be added right next to the field.

      1. SOLR-3668.patch
        1 kB
        Stefan Matheis (steffkes)

        Activity

        Hide
        Stefan Matheis (steffkes) added a comment -

        Thanks for Reporting this Chantal, the code you're looking for is located in dataimport.js on Line 531-535

        var custom_parameters = $( '#custom_parameters', form ).val();
        if( custom_parameters.length )
        {
          array.push( { name : 'custom_parameters', value: custom_parameters } );
        }

        and that is also already the reason, why it doesn't work. comparing a direct request, which logs this:

        INFO: [collection1] webapp=/solr path=/dataimport params={command=full-import&importfile=/absolute/path/to/file.xml} status=0 QTime=1

        against an UI-based Request, which logs that:

        INFO: [collection1] webapp=/solr path=/dataimport params={command=full-import&custom_parameters=importfile%3D/absolute/path/to/file.xml} status=0 QTime=3

        will try to fix that tomorrow. perhaps there is a quick chance to inject the raw string directly into the url .. otherwise i'll split the string up first and let jquery compose the final one - will see what's possible here.

        If it would work by specifying the parameter in a different way it would be great if a little info would be added right next to the field.

        i guess the information/hint would be helpful in either case, will integrate one

        Show
        Stefan Matheis (steffkes) added a comment - Thanks for Reporting this Chantal, the code you're looking for is located in dataimport.js on Line 531-535 var custom_parameters = $( '#custom_parameters', form ).val(); if ( custom_parameters.length ) { array.push( { name : 'custom_parameters', value: custom_parameters } ); } and that is also already the reason, why it doesn't work. comparing a direct request, which logs this: INFO: [collection1] webapp=/solr path=/dataimport params={command=full- import &importfile=/absolute/path/to/file.xml} status=0 QTime=1 against an UI-based Request, which logs that: INFO: [collection1] webapp=/solr path=/dataimport params={command=full- import &custom_parameters=importfile%3D/absolute/path/to/file.xml} status=0 QTime=3 will try to fix that tomorrow. perhaps there is a quick chance to inject the raw string directly into the url .. otherwise i'll split the string up first and let jquery compose the final one - will see what's possible here. If it would work by specifying the parameter in a different way it would be great if a little info would be added right next to the field. i guess the information/hint would be helpful in either case, will integrate one
        Hide
        Chantal Ackermann added a comment -

        Hi Stefan,

        thanks for fixing! I had misconfigured my DIH at that time, and it did not output the INFO messages you posted.
        I'm importing directly via /dataimport - so no hurry from my part.

        C

        Show
        Chantal Ackermann added a comment - Hi Stefan, thanks for fixing! I had misconfigured my DIH at that time, and it did not output the INFO messages you posted. I'm importing directly via /dataimport - so no hurry from my part. C
        Hide
        Stefan Matheis (steffkes) added a comment -

        Chantal, would you mind applying this one and have another look at it?

        Show
        Stefan Matheis (steffkes) added a comment - Chantal, would you mind applying this one and have another look at it?
        Hide
        Stefan Matheis (steffkes) added a comment -

        Committed revision 1381518. trunk
        Committed revision 1381520. 4x

        Show
        Stefan Matheis (steffkes) added a comment - Committed revision 1381518. trunk Committed revision 1381520. 4x
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] Stefan Matheis
        http://svn.apache.org/viewvc?view=revision&revision=1381524

        SOLR-3668: fix changes entry, it's a bugfix

        Show
        Commit Tag Bot added a comment - [branch_4x commit] Stefan Matheis http://svn.apache.org/viewvc?view=revision&revision=1381524 SOLR-3668 : fix changes entry, it's a bugfix
        Hide
        Commit Tag Bot added a comment -

        [branch_4x commit] Stefan Matheis
        http://svn.apache.org/viewvc?view=revision&revision=1381520

        SOLR-3668: fix handling of custom parameters for dataimport on admin ui

        Show
        Commit Tag Bot added a comment - [branch_4x commit] Stefan Matheis http://svn.apache.org/viewvc?view=revision&revision=1381520 SOLR-3668 : fix handling of custom parameters for dataimport on admin ui

          People

          • Assignee:
            Stefan Matheis (steffkes)
            Reporter:
            Chantal Ackermann
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development