Solr
  1. Solr
  2. SOLR-125

disambiguate NamedList for JSON output

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2
    • Component/s: search
    • Labels:
      None

      Description

      A NamedList implies both ordered access and access by name. For JSON/Ruby/Python, we normally need to pick one or the other as maps are not ordered by default in most clients.

        Issue Links

          Activity

          Hide
          Yonik Seeley added a comment -

          Preliminary patch attached.

          General approach: NamedList needs to get off the fence and say if order or access-by-key is more important.

          Starting with the assumption that NamedList==ordered, I subclassed it to create a SimpleOrderedMap.
          This has the advantage of not having to change any method signatures anywhere since a SimpleOrderedMap is a NamedList. It also keeps order for clients who can maintain order, and is much more efficient than something like the alternative: LinkedHashMap.

          XML output will be unchanged.

          JSON (and friends) output will be a map (JSON object) for SimpleOrderedMap, or (by default) a "flat" array, alternating key and value. The json.nl param can be used to change this default for NamedLists.

          Most instances of "new NamedList()" should be changed to SimpleOrderedMap. Items that are sorted, where order is more important (like field facets) should continue to use NamedList.

          Show
          Yonik Seeley added a comment - Preliminary patch attached. General approach: NamedList needs to get off the fence and say if order or access-by-key is more important. Starting with the assumption that NamedList==ordered, I subclassed it to create a SimpleOrderedMap. This has the advantage of not having to change any method signatures anywhere since a SimpleOrderedMap is a NamedList. It also keeps order for clients who can maintain order, and is much more efficient than something like the alternative: LinkedHashMap. XML output will be unchanged. JSON (and friends) output will be a map (JSON object) for SimpleOrderedMap, or (by default) a "flat" array, alternating key and value. The json.nl param can be used to change this default for NamedLists. Most instances of "new NamedList()" should be changed to SimpleOrderedMap. Items that are sorted, where order is more important (like field facets) should continue to use NamedList.
          Hide
          Yonik Seeley added a comment -

          I've locally made some changes to DumpRequestHandler...
          1) changed the streams from a map of map to an array of map
          2) directly add the context map instead of iterating over it

          Any reason not to do #2? The outer NamedList previously just had each inner map under a key of "stream".

          public class DumpRequestHandler extends RequestHandlerBase
          {
          @Override
          public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp) throws IOException
          {
          // Show params
          rsp.add( "params", req.getParams().toNamedList() );

          // Write the streams...
          if( req.getContentStreams() != null ) {
          ArrayList streams = new ArrayList();
          // Cycle through each stream
          for( ContentStream content : req.getContentStreams() )

          { NamedList<Object> stream = new SimpleOrderedMap<Object>(); stream.add( "name", content.getName() ); stream.add( "fieldName", content.getSourceInfo() ); stream.add( "size", content.getSize() ); stream.add( "contentType", content.getContentType() ); stream.add( "stream", IOUtils.toString( content.getStream() ) ); streams.add( stream ); }

          rsp.add( "streams", streams );
          }

          rsp.add("context", req.getContext());
          }

          Sample output:
          [
          "responseHeader",

          { "status":0, "QTime":0}

          ,
          "params",

          { "wt":"json", "indent":"on", "stream.body":[ "wow", "dude"]}

          ,
          "streams",[

          { "name":null, "fieldName":"stream.body", "size":null, "contentType":null, "stream":"wow"}

          ,

          { "name":null, "fieldName":"stream.body", "size":null, "contentType":null, "stream":"dude"}

          ],
          "context",{}]

          Show
          Yonik Seeley added a comment - I've locally made some changes to DumpRequestHandler... 1) changed the streams from a map of map to an array of map 2) directly add the context map instead of iterating over it Any reason not to do #2? The outer NamedList previously just had each inner map under a key of "stream". public class DumpRequestHandler extends RequestHandlerBase { @Override public void handleRequestBody(SolrQueryRequest req, SolrQueryResponse rsp) throws IOException { // Show params rsp.add( "params", req.getParams().toNamedList() ); // Write the streams... if( req.getContentStreams() != null ) { ArrayList streams = new ArrayList(); // Cycle through each stream for( ContentStream content : req.getContentStreams() ) { NamedList<Object> stream = new SimpleOrderedMap<Object>(); stream.add( "name", content.getName() ); stream.add( "fieldName", content.getSourceInfo() ); stream.add( "size", content.getSize() ); stream.add( "contentType", content.getContentType() ); stream.add( "stream", IOUtils.toString( content.getStream() ) ); streams.add( stream ); } rsp.add( "streams", streams ); } rsp.add("context", req.getContext()); } Sample output: [ "responseHeader", { "status":0, "QTime":0} , "params", { "wt":"json", "indent":"on", "stream.body":[ "wow", "dude"]} , "streams",[ { "name":null, "fieldName":"stream.body", "size":null, "contentType":null, "stream":"wow"} , { "name":null, "fieldName":"stream.body", "size":null, "contentType":null, "stream":"dude"} ], "context",{}]
          Hide
          Yonik Seeley added a comment -

          Sample output of facets w/ JSON and the "flat" default for NamedLists:

          "facet_fields":

          { "cat":[ "electronics",14, "memory",3, "card",2, "connector",2, "drive",2, "graphics",2, "hard",2, "monitor",2, "search",2, "software",2, "camera",1, "copier",1, "multifunction",1, "music",1, "printer",1, "scanner",1]}

          }}

          Show
          Yonik Seeley added a comment - Sample output of facets w/ JSON and the "flat" default for NamedLists: "facet_fields": { "cat":[ "electronics",14, "memory",3, "card",2, "connector",2, "drive",2, "graphics",2, "hard",2, "monitor",2, "search",2, "software",2, "camera",1, "copier",1, "multifunction",1, "music",1, "printer",1, "scanner",1]} }}
          Hide
          Yonik Seeley added a comment -

          OK, I've committed this. I wanted to get it out of the way since it touched a number of files in a minor way (keeps other patches from applying cleanly).

          json.nl=flat default is subjective... if people think a different default is better, let's discuss now.

          Show
          Yonik Seeley added a comment - OK, I've committed this. I wanted to get it out of the way since it touched a number of files in a minor way (keeps other patches from applying cleanly). json.nl=flat default is subjective... if people think a different default is better, let's discuss now.
          Hide
          Hoss Man added a comment -

          This bug was modified as part of a bulk update using the criteria...

          • Marked ("Resolved" or "Closed") and "Fixed"
          • Had no "Fix Version" versions
          • Was listed in the CHANGES.txt for 1.2

          The Fix Version for all 39 issues found was set to 1.2, email notification
          was suppressed to prevent excessive email.

          For a list of all the issues modified, search jira comments for this
          (hopefully) unique string: 20080415hossman2

          Show
          Hoss Man added a comment - This bug was modified as part of a bulk update using the criteria... Marked ("Resolved" or "Closed") and "Fixed" Had no "Fix Version" versions Was listed in the CHANGES.txt for 1.2 The Fix Version for all 39 issues found was set to 1.2, email notification was suppressed to prevent excessive email. For a list of all the issues modified, search jira comments for this (hopefully) unique string: 20080415hossman2

            People

            • Assignee:
              Unassigned
              Reporter:
              Yonik Seeley
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development