Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-11598

ExportWriter should support sorting on more than 4 sort fields

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 7.5, 8.0
    • Component/s: streaming expressions
    • Labels:

      Description

      I am a user of Streaming and I am currently trying to use rollups on an 10 dimensional document.
      I am unable to get correct results on this query as I am bounded by the limitation of the export handler which supports only 4 sort fields.
      I do not see why this needs to be the case, as it could very well be 10 or 20.
      My current needs would be satisfied with 10, but one would want to ask why can't it be any decent integer n, beyond which we know performance degrades, but even then it should be caveat emptor.

      [~varunthacker]

      Code Link:
      https://github.com/apache/lucene-solr/blob/19db1df81a18e6eb2cce5be973bf2305d606a9f8/solr/core/src/java/org/apache/solr/handler/ExportWriter.java#L455

      Error
      null:java.io.IOException: A max of 4 sorts can be specified
      at org.apache.solr.handler.ExportWriter.getSortDoc(ExportWriter.java:452)
      at org.apache.solr.handler.ExportWriter.writeDocs(ExportWriter.java:228)
      at org.apache.solr.handler.ExportWriter.lambda$null$1(ExportWriter.java:219)
      at org.apache.solr.common.util.JavaBinCodec.writeIterator(JavaBinCodec.java:664)
      at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:333)
      at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:223)
      at org.apache.solr.common.util.JavaBinCodec$1.put(JavaBinCodec.java:394)
      at org.apache.solr.handler.ExportWriter.lambda$null$2(ExportWriter.java:219)
      at org.apache.solr.common.util.JavaBinCodec.writeMap(JavaBinCodec.java:437)
      at org.apache.solr.common.util.JavaBinCodec.writeKnownType(JavaBinCodec.java:354)
      at org.apache.solr.common.util.JavaBinCodec.writeVal(JavaBinCodec.java:223)
      at org.apache.solr.common.util.JavaBinCodec$1.put(JavaBinCodec.java:394)
      at org.apache.solr.handler.ExportWriter.lambda$write$3(ExportWriter.java:217)
      at org.apache.solr.common.util.JavaBinCodec.writeMap(JavaBinCodec.java:437)
      at org.apache.solr.handler.ExportWriter.write(ExportWriter.java:215)
      at org.apache.solr.core.SolrCore$3.write(SolrCore.java:2601)
      at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:49)
      at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:809)
      at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:538)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:361)
      at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:305)
      at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1691)
      at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:582)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
      at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
      at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
      at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1180)
      at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:512)
      at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
      at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1112)
      at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
      at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
      at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
      at org.eclipse.jetty.rewrite.handler.RewriteHandler.handle(RewriteHandler.java:335)
      at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
      at org.eclipse.jetty.server.Server.handle(Server.java:534)
      at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:320)
      at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:251)
      at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
      at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
      at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.executeProduceConsume(ExecuteProduceConsume.java:303)
      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceConsume(ExecuteProduceConsume.java:148)
      at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:136)
      at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:671)
      at org.eclipse.jetty.util.thread.QueuedThreadPool$2.run(QueuedThreadPool.java:589)
      at java.lang.Thread.run(Thread.java:745)

        Attachments

        1. streaming-export reports.xlsx
          5 kB
          Amrit Sarkar
        2. SOLR-11598-master.patch
          22 kB
          Amrit Sarkar
        3. SOLR-11598-6_6-streamtests
          30 kB
          Amrit Sarkar
        4. SOLR-11598-6_6.patch
          22 kB
          Amrit Sarkar
        5. SOLR-11598.patch
          14 kB
          Amrit Sarkar
        6. SOLR-11598.patch
          15 kB
          Amrit Sarkar
        7. SOLR-11598.patch
          17 kB
          Amrit Sarkar
        8. SOLR-11598.patch
          17 kB
          Amrit Sarkar
        9. SOLR-11598.patch
          119 kB
          Varun Thacker
        10. SOLR-11598.patch
          154 kB
          Amrit Sarkar
        11. SOLR-11598.patch
          149 kB
          Varun Thacker
        12. SOLR-11598.patch
          149 kB
          Varun Thacker
        13. SOLR-11598.patch
          150 kB
          Varun Thacker
        14. SOLR-11598.patch
          147 kB
          Amrit Sarkar
        15. SOLR-11598.patch
          151 kB
          Varun Thacker

          Activity

            People

            • Assignee:
              varun Varun Thacker
              Reporter:
              aroopganguly Aroop
            • Votes:
              0 Vote for this issue
              Watchers:
              13 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: