Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 4.1
    • Labels:
      None

      Description

      When I start Solr with the attached conf directory, and hit the Schema Browser, the loading circle spins permanently.

      I don't know if the problem is in the UI or in Solr. The UI does not display the Ajax solr calls, and I don't have a debugging proxy.

      1. SOLR-3734_schema_browser_blocks_solr_conf_dir.zip
        140 kB
        Lance Norskog
      2. SOLR-3734.patch
        6 kB
        Stefan Matheis (steffkes)
      3. SOLR-3734.patch
        4 kB
        Stefan Matheis (steffkes)

        Issue Links

          Activity

          Hide
          Stefan Matheis (steffkes) added a comment -

          The reason is pretty simple: in your schema, you have copyfield for a non-existing field defined:

          alphaNameSort: {
            type: "alphaOnlySort",
            flags: "IT-----O-----l",
            copyDests: [ ],
            copySources: [
              "org.apache.solr.schema.SchemaField:name_def{type=textgen,properties=indexed,tokenized,stored}"
            ]
          }

          from the schema.xml:

           <!-- copy name to alphaNameSort, a field designed for sorting by name -->
           <copyField source="name_def" dest="alphaNameSort"/>
           <copyField source="name_en" dest="alphaNameSort_en"/>

          That is a case which the Schema-Browser is right now not able to handle correctly .. Perhaps we can/should collect such information/"problems" and display them to the user?

          Show
          Stefan Matheis (steffkes) added a comment - The reason is pretty simple: in your schema, you have copyfield for a non-existing field defined: alphaNameSort: { type: "alphaOnlySort" , flags: "IT-----O-----l" , copyDests: [ ], copySources: [ "org.apache.solr.schema.SchemaField:name_def{type=textgen,properties=indexed,tokenized,stored}" ] } from the schema.xml: <!-- copy name to alphaNameSort, a field designed for sorting by name --> <copyField source= "name_def" dest= "alphaNameSort" /> <copyField source= "name_en" dest= "alphaNameSort_en" /> That is a case which the Schema-Browser is right now not able to handle correctly .. Perhaps we can/should collect such information/"problems" and display them to the user?
          Hide
          Lance Norskog added a comment -

          If there is an exception or a timeout, the UI should show the problem.

          Show
          Lance Norskog added a comment - If there is an exception or a timeout, the UI should show the problem.
          Hide
          Stefan Matheis (steffkes) added a comment -

          First Draft, just a quick hack.

          Show
          Stefan Matheis (steffkes) added a comment - First Draft, just a quick hack.
          Hide
          Hoss Man added a comment -

          The reason is pretty simple: in your schema, you have copyfield for a non-existing field defined:

          steffkes: i don't understand your comment. those copyField's are legal, and the fields do exist, because there are dynamicField's that define them...

          <dynamicField name="*_def" type="textgen" indexed="true"/>
          <dynamicField name="*_en" type="text_en" indexed="true"/>
          

          Is there a problem in the request handler the UI is using to get schema information, that it doesn't expose these source/dest fields if they xist because of dynamicField, or is the problem solely in the javascript code?

          (ie: can i help?)

          Show
          Hoss Man added a comment - The reason is pretty simple: in your schema, you have copyfield for a non-existing field defined: steffkes: i don't understand your comment. those copyField's are legal, and the fields do exist, because there are dynamicField's that define them... <dynamicField name="*_def" type="textgen" indexed="true"/> <dynamicField name="*_en" type="text_en" indexed="true"/> Is there a problem in the request handler the UI is using to get schema information, that it doesn't expose these source/dest fields if they xist because of dynamicField, or is the problem solely in the javascript code? (ie: can i help?)
          Hide
          Hoss Man added a comment -

          watching the logs while using the UI, it seems that the crux of the info all comes from these two urls...

          INFO: [collection1] webapp=/solr path=/admin/luke params={numTerms=0&wt=json} status=0 QTime=10 
          Sep 5, 2012 4:51:45 PM org.apache.solr.core.SolrCore execute
          INFO: [collection1] webapp=/solr path=/admin/luke params={wt=json&show=schema} status=0 QTime=3 
          

          Ignoring for a moment that it seems absurd to me that the strings in the "copyDests" and "copySources" arrays are the "toString" of the SchemaField/CopyField objects, instead of just the field names: i think the data in the response is correct.

          It seems, if i'm understanding correctly, that the root problem is that the UI expects all copySources returned by show=schema to be real field names that were already by the previous numTerms=0 request - but this is not guaranteed, nor is it abnormal to have these copyFields based on dynamicFields.

          And if i'm understanding correctly, all the UI is doing with this information is populating "Copied from" and "Copied to" lists, which then link to "/schema-browser?field=foo" (and maybe "/schema-browser?dynamic-field=foo_*" - but i didn't actually see any examples of that) to see details about those fields. But the "/schema-browser?field=foo" URLs only seem to work if that field actaully exists in the luke?numTerms=0 response – even if they are legal because of dynamicField.

          So, for example, this address_s URL doesn't work in the UI using the example configs on trunk if you have no data indexed, but once you index the sample data it does start to work...

          http://localhost:8983/solr/#/collection1/schema-browser?field=address_s

          I think the "correct" fixes would be...

          • the UI should not freak out if a copySources or copyDest refers to a field or dynamic field pattern it doesn't know about. it should list it in the display as usual and provide the appropriate ?field=foo or ?field=foo_* link (depending on wether the copySource contains a wildcard.
          • if someone clicks on one of those links, and there is no field data available about a field (because it isn't explicitly declared, nor does it exist in the index, so it's not returned by the luke?numTerms=0 request) then the UI should display a simple warning "Field not found in index, no details available" or something like that.

          We can probably look into improving that second case in the future (so even if it's not in the index, and wasn't returned by the luke?numTerms-0 request, the UI should be able to ask luke for info about a field by name, and have luke response back with "not currently in the index, but it is a legal field name because of dynamicField foo_*, and heres' the fieldType and field flags")

          Show
          Hoss Man added a comment - watching the logs while using the UI, it seems that the crux of the info all comes from these two urls... INFO: [collection1] webapp=/solr path=/admin/luke params={numTerms=0&wt=json} status=0 QTime=10 Sep 5, 2012 4:51:45 PM org.apache.solr.core.SolrCore execute INFO: [collection1] webapp=/solr path=/admin/luke params={wt=json&show=schema} status=0 QTime=3 Ignoring for a moment that it seems absurd to me that the strings in the "copyDests" and "copySources" arrays are the "toString" of the SchemaField/CopyField objects, instead of just the field names: i think the data in the response is correct. It seems, if i'm understanding correctly, that the root problem is that the UI expects all copySources returned by show=schema to be real field names that were already by the previous numTerms=0 request - but this is not guaranteed, nor is it abnormal to have these copyFields based on dynamicFields. And if i'm understanding correctly, all the UI is doing with this information is populating "Copied from" and "Copied to" lists, which then link to "/schema-browser?field=foo" (and maybe "/schema-browser?dynamic-field=foo_*" - but i didn't actually see any examples of that) to see details about those fields. But the "/schema-browser?field=foo" URLs only seem to work if that field actaully exists in the luke?numTerms=0 response – even if they are legal because of dynamicField. So, for example, this address_s URL doesn't work in the UI using the example configs on trunk if you have no data indexed, but once you index the sample data it does start to work... http://localhost:8983/solr/#/collection1/schema-browser?field=address_s I think the "correct" fixes would be... the UI should not freak out if a copySources or copyDest refers to a field or dynamic field pattern it doesn't know about. it should list it in the display as usual and provide the appropriate ?field=foo or ?field=foo_* link (depending on wether the copySource contains a wildcard. if someone clicks on one of those links, and there is no field data available about a field (because it isn't explicitly declared, nor does it exist in the index, so it's not returned by the luke?numTerms=0 request) then the UI should display a simple warning "Field not found in index, no details available" or something like that. We can probably look into improving that second case in the future (so even if it's not in the index, and wasn't returned by the luke?numTerms-0 request, the UI should be able to ask luke for info about a field by name, and have luke response back with "not currently in the index, but it is a legal field name because of dynamicField foo_*, and heres' the fieldType and field flags")
          Hide
          Hoss Man added a comment -

          Ignoring for a moment that it seems absurd to me that the strings in the "copyDests" and "copySources" arrays are the "toString" of the SchemaField/CopyField objects, instead of just the field names: i think the data in the response is correct.

          spun this off into SOLR-3795

          Show
          Hoss Man added a comment - Ignoring for a moment that it seems absurd to me that the strings in the "copyDests" and "copySources" arrays are the "toString" of the SchemaField/CopyField objects, instead of just the field names: i think the data in the response is correct. spun this off into SOLR-3795
          Hide
          Stefan Matheis (steffkes) added a comment -

          It seems, if i'm understanding correctly, that the root problem is that the UI expects all copySources returned by show=schema to be real field names that were already by the previous numTerms=0 request - but this is not guaranteed, nor is it abnormal to have these copyFields based on dynamicFields.

          steffkes 0 : hoss 1 :/ exactly what you said - didn't look close enough .. i'll rework the patch and try to get your two points in it!

          Show
          Stefan Matheis (steffkes) added a comment - It seems, if i'm understanding correctly, that the root problem is that the UI expects all copySources returned by show=schema to be real field names that were already by the previous numTerms=0 request - but this is not guaranteed, nor is it abnormal to have these copyFields based on dynamicFields. steffkes 0 : hoss 1 :/ exactly what you said - didn't look close enough .. i'll rework the patch and try to get your two points in it!
          Hide
          Stefan Matheis (steffkes) added a comment -

          Next version ..

          • removing Legacy-Code for Handling CopyDests/CopySources
          • Introducing partial Marker, used to display a info-box
          • Hiding all Informations which aren't available while in partial-state
          Show
          Stefan Matheis (steffkes) added a comment - Next version .. removing Legacy-Code for Handling CopyDests/CopySources Introducing partial Marker, used to display a info-box Hiding all Informations which aren't available while in partial-state
          Hide
          Stefan Matheis (steffkes) added a comment -

          Lance Norskog would you mind having a look, if that works as expected?

          Show
          Stefan Matheis (steffkes) added a comment - Lance Norskog would you mind having a look, if that works as expected?
          Hide
          Lance Norskog added a comment -

          Hi-

          Yes, this works in branch_4x, using the schema I submitted. I do not have ability to test whether it handles exceptions well. When you are writing new analyzer components, it is helpful for the UI to say "your code blew up".

          Show
          Lance Norskog added a comment - Hi- Yes, this works in branch_4x, using the schema I submitted. I do not have ability to test whether it handles exceptions well. When you are writing new analyzer components, it is helpful for the UI to say "your code blew up".
          Hide
          Stefan Matheis (steffkes) added a comment -

          Committed revision 1392318. trunk
          Committed revision 1392320. branch_4x

          Show
          Stefan Matheis (steffkes) added a comment - Committed revision 1392318. trunk Committed revision 1392320. branch_4x
          Hide
          Stefan Matheis (steffkes) added a comment -

          Committed revision 1394980. lucene_solr_4_0

          Show
          Stefan Matheis (steffkes) added a comment - Committed revision 1394980. lucene_solr_4_0
          Hide
          Commit Tag Bot added a comment -

          [branch_4x commit] Stefan Matheis
          http://svn.apache.org/viewvc?view=revision&revision=1392320

          SOLR-3734: Improve Schema-Browser Handling for CopyField using dynamicField's (merge r1392318)

          Show
          Commit Tag Bot added a comment - [branch_4x commit] Stefan Matheis http://svn.apache.org/viewvc?view=revision&revision=1392320 SOLR-3734 : Improve Schema-Browser Handling for CopyField using dynamicField's (merge r1392318)

            People

            • Assignee:
              Stefan Matheis (steffkes)
              Reporter:
              Lance Norskog
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development