Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9141

ClassCastException occurs in /sql handler with GROUP BY aggregationMode=facet and single shard

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 6.0
    • Fix Version/s: None
    • Component/s: Parallel SQL
    • Labels:
      None

      Description

      ClassCastException occurs in /sql request handler using ORDER BY GROUP BY clause.

      $ curl --data-urlencode "stmt=select count(*) from access_log" "http://localhost:8983/solr/access_log/sql?aggregationMode=facet"
      {"result-set":{"docs":[
      {"count(*)":1309},
      {"EOF":true,"RESPONSE_TIME":239}]}}
      
      $ curl --data-urlencode 'stmt=select response, count(*) as count from access_log group by response' "http://localhost:8983/solr/access_log/sql?aggregationMode=facet"
      {"result-set":{"docs":[
      {"EXCEPTION":"java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long","EOF":true,"RESPONSE_TIME":53}]}}
      

      See following error messages:

      2016-05-19 10:18:06.477 ERROR (qtp1791930789-21) [c:access_log s:shard1 r:core_node1 x:access_log_shard1_replica1] o.a.s.c.s.i.s.ExceptionStream java.io.IOException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
          at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:300)
          at org.apache.solr.handler.SQLHandler$LimitStream.open(SQLHandler.java:1265)
          at org.apache.solr.client.solrj.io.stream.SelectStream.open(SelectStream.java:153)
          at org.apache.solr.handler.SQLHandler$MetadataStream.open(SQLHandler.java:1511)
          at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:47)
          at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:362)
          at org.apache.solr.response.TextResponseWriter.writeTupleStream(TextResponseWriter.java:301)
          at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:167)
          at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:183)
          at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:299)
          at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:95)
          at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:60)
          at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
          at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:725)
          at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
          at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
          at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:308)
          at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:262)
          at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
          at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
          at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
          at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
          at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
          at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
          at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
          at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
          at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
          at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
          at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
          at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
          at org.eclipse.jetty.server.Server.handle(Server.java:518)
          at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
          at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
          at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
          at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
          at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
          at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
          at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
          at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
          at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
          at java.lang.Thread.run(Thread.java:745)
      Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
          at org.apache.solr.client.solrj.io.stream.FacetStream.fillTuples(FacetStream.java:461)
          at org.apache.solr.client.solrj.io.stream.FacetStream.getTuples(FacetStream.java:420)
          at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:297)
          ... 43 more
      
      1. SOLR-9141.patch
        9 kB
        James Dyer
      2. SOLR-9141.patch
        9 kB
        James Dyer
      3. SOLR-9141.patch
        1 kB
        Minoru Osuka
      4. SOLR-9141-test.patch
        9 kB
        James Dyer
      5. SOLR-9141-test.patch
        1 kB
        James Dyer

        Activity

        Hide
        minoru Minoru Osuka added a comment -

        Attach path file.

        Show
        minoru Minoru Osuka added a comment - Attach path file.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        I don't see an ORDER BY clause in your query. It looks like it's failing on a basic GROUP BY?

        I'll have to see if I can reproduce this with a test. What is the field type for the response column?

        Show
        joel.bernstein Joel Bernstein added a comment - I don't see an ORDER BY clause in your query. It looks like it's failing on a basic GROUP BY? I'll have to see if I can reproduce this with a test. What is the field type for the response column?
        Hide
        minoru Minoru Osuka added a comment - - edited

        Hi Joel,

        Yes, It seems basic ORDER BY GROUP BY clause is failing.

        Field type for the response is following:

        $ curl "http://localhost:8983/solr/access_log/schema/fields/response"
        {
          "responseHeader":{
            "status":0,
            "QTime":3},
          "field":{
            "name":"response",
            "type":"int",
            "docValues":true,
            "multiValued":false,
            "indexed":true,
            "stored":true}}
        
        $ curl "http://localhost:8983/solr/access_log/schema/fieldtypes/int"
        {
          "responseHeader":{
            "status":0,
            "QTime":246},
          "fieldType":{
            "name":"int",
            "class":"solr.TrieIntField",
            "positionIncrementGap":"0",
            "precisionStep":"0",
            "fields":["response"],
            "dynamicFields":["*_i"]},
          "warn":"This API is deprecated"}
        

        aggregationMode=map_reduce works fine.
        Rsult of simple count is following:

        $ curl --data-urlencode "stmt=SELECT count(*) FROM access_log" "http://localhost:8983/solr/access_log/sql?aggregationMode=map_reduce"
        {"result-set":{"docs":[
        {"count(*)":1157},
        {"EOF":true,"RESPONSE_TIME":7}]}}
        

        And result of ORDER BY GROUP BY clause is following:

        $ curl --data-urlencode 'stmt=SELECT response, count(*) FROM access_log GROUP BY response' "http://localhost:8983/solr/access_log/sql?aggregationMode=map_reduce"
        {"result-set":{"docs":[
        {"response":200},
        {"response":404},
        {"response":500},
        {"EOF":true,"RESPONSE_TIME":18}]}}
        

        Also aggregationMode=facet works fine without ORDER BY GROUP BY clause.

        $ curl --data-urlencode "stmt=SELECT count(*) FROM access_log" "http://localhost:8983/solr/access_log/sql?aggregationMode=facet"
        {"result-set":{"docs":[
        {"count(*)":1157},
        {"EOF":true,"RESPONSE_TIME":9}]}}
        

        If SQL specified ORDER BY GROUP BY clause, it does't work due to ClassCastException.

        $ curl --data-urlencode 'stmt=SELECT response, count(*) FROM access_log GROUP BY response' "http://localhost:8983/solr/access_log/sql?aggregationMode=facet"
        {"result-set":{"docs":[
        {"EXCEPTION":"java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long","EOF":true,"RESPONSE_TIME":35}]}}
        

        Please see following solr.log:

        2016-05-24 08:48:24.944 INFO  (qtp1791930789-21) [c:access_log s:shard1 r:core_node1 x:access_log_shard1_replica1] o.a.s.c.S.Request [access_log_shard1_replica1]  webapp=/solr path=/sql params={aggregationMode=facet&stmt=select+response,+count(*)+as+count+from+access_log+group+by+response} status=0 QTime=5
        2016-05-24 08:48:24.970 INFO  (qtp1791930789-12) [c:access_log s:shard1 r:core_node1 x:access_log_shard1_replica1] o.a.s.c.S.Request [access_log_shard1_replica1]  webapp=/solr path=/select params={q=*:*&json.facet={"response":{"type":"terms","field":"response","limit":100,"sort":{"index":"asc"},"facet":{}}}&_stateVer_=access_log:10&rows=0&wt=javabin&version=2} hits=1157 status=0 QTime=5
        2016-05-24 08:48:24.979 ERROR (qtp1791930789-21) [c:access_log s:shard1 r:core_node1 x:access_log_shard1_replica1] o.a.s.c.s.i.s.ExceptionStream java.io.IOException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
            at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:300)
            at org.apache.solr.client.solrj.io.stream.SelectStream.open(SelectStream.java:153)
            at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:47)
            at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:362)
            at org.apache.solr.response.TextResponseWriter.writeTupleStream(TextResponseWriter.java:301)
            at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:167)
            at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:183)
            at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:299)
            at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:95)
            at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:60)
            at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65)
            at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:725)
            at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469)
            at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229)
            at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676)
            at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:308)
            at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:262)
            at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668)
            at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143)
            at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548)
            at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226)
            at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160)
            at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511)
            at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185)
            at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092)
            at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141)
            at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213)
            at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119)
            at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134)
            at org.eclipse.jetty.server.Server.handle(Server.java:518)
            at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308)
            at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244)
            at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273)
            at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95)
            at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93)
            at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246)
            at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156)
            at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654)
            at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572)
            at java.lang.Thread.run(Thread.java:745)
        Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
            at org.apache.solr.client.solrj.io.stream.FacetStream.fillTuples(FacetStream.java:461)
            at org.apache.solr.client.solrj.io.stream.FacetStream.getTuples(FacetStream.java:420)
            at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:297)
            ... 41 more
        

        After this patch applied, ORDER BY GROUP BY clause works fine using aggregationMode=facet.

        $ curl --data-urlencode 'stmt=SELECT response, count(*) FROM access_log GROUP BY response' "http://localhost:8983/solr/access_log/sql?aggregationMode=facet"
        {"result-set":{"docs":[
        {"response":200,"count(*)":1153},
        {"response":404,"count(*)":3},
        {"response":500,"count(*)":1},
        {"EOF":true,"RESPONSE_TIME":9}]}}
        
        Show
        minoru Minoru Osuka added a comment - - edited Hi Joel, Yes, It seems basic ORDER BY GROUP BY clause is failing. Field type for the response is following: $ curl "http://localhost:8983/solr/access_log/schema/fields/response" { "responseHeader":{ "status":0, "QTime":3}, "field":{ "name":"response", "type":"int", "docValues":true, "multiValued":false, "indexed":true, "stored":true}} $ curl "http://localhost:8983/solr/access_log/schema/fieldtypes/int" { "responseHeader":{ "status":0, "QTime":246}, "fieldType":{ "name":"int", "class":"solr.TrieIntField", "positionIncrementGap":"0", "precisionStep":"0", "fields":["response"], "dynamicFields":["*_i"]}, "warn":"This API is deprecated"} aggregationMode=map_reduce works fine. Rsult of simple count is following: $ curl --data-urlencode "stmt=SELECT count(*) FROM access_log" "http://localhost:8983/solr/access_log/sql?aggregationMode=map_reduce" {"result-set":{"docs":[ {"count(*)":1157}, {"EOF":true,"RESPONSE_TIME":7}]}} And result of ORDER BY GROUP BY clause is following: $ curl --data-urlencode 'stmt=SELECT response, count(*) FROM access_log GROUP BY response' "http://localhost:8983/solr/access_log/sql?aggregationMode=map_reduce" {"result-set":{"docs":[ {"response":200}, {"response":404}, {"response":500}, {"EOF":true,"RESPONSE_TIME":18}]}} Also aggregationMode=facet works fine without ORDER BY GROUP BY clause. $ curl --data-urlencode "stmt=SELECT count(*) FROM access_log" "http://localhost:8983/solr/access_log/sql?aggregationMode=facet" {"result-set":{"docs":[ {"count(*)":1157}, {"EOF":true,"RESPONSE_TIME":9}]}} If SQL specified ORDER BY GROUP BY clause, it does't work due to ClassCastException. $ curl --data-urlencode 'stmt=SELECT response, count(*) FROM access_log GROUP BY response' "http://localhost:8983/solr/access_log/sql?aggregationMode=facet" {"result-set":{"docs":[ {"EXCEPTION":"java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long","EOF":true,"RESPONSE_TIME":35}]}} Please see following solr.log: 2016-05-24 08:48:24.944 INFO (qtp1791930789-21) [c:access_log s:shard1 r:core_node1 x:access_log_shard1_replica1] o.a.s.c.S.Request [access_log_shard1_replica1] webapp=/solr path=/sql params={aggregationMode=facet&stmt=select+response,+count(*)+as+count+from+access_log+group+by+response} status=0 QTime=5 2016-05-24 08:48:24.970 INFO (qtp1791930789-12) [c:access_log s:shard1 r:core_node1 x:access_log_shard1_replica1] o.a.s.c.S.Request [access_log_shard1_replica1] webapp=/solr path=/select params={q=*:*&json.facet={"response":{"type":"terms","field":"response","limit":100,"sort":{"index":"asc"},"facet":{}}}&_stateVer_=access_log:10&rows=0&wt=javabin&version=2} hits=1157 status=0 QTime=5 2016-05-24 08:48:24.979 ERROR (qtp1791930789-21) [c:access_log s:shard1 r:core_node1 x:access_log_shard1_replica1] o.a.s.c.s.i.s.ExceptionStream java.io.IOException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:300) at org.apache.solr.client.solrj.io.stream.SelectStream.open(SelectStream.java:153) at org.apache.solr.client.solrj.io.stream.ExceptionStream.open(ExceptionStream.java:47) at org.apache.solr.handler.StreamHandler$TimerStream.open(StreamHandler.java:362) at org.apache.solr.response.TextResponseWriter.writeTupleStream(TextResponseWriter.java:301) at org.apache.solr.response.TextResponseWriter.writeVal(TextResponseWriter.java:167) at org.apache.solr.response.JSONWriter.writeNamedListAsMapWithDups(JSONResponseWriter.java:183) at org.apache.solr.response.JSONWriter.writeNamedList(JSONResponseWriter.java:299) at org.apache.solr.response.JSONWriter.writeResponse(JSONResponseWriter.java:95) at org.apache.solr.response.JSONResponseWriter.write(JSONResponseWriter.java:60) at org.apache.solr.response.QueryResponseWriterUtil.writeQueryResponse(QueryResponseWriterUtil.java:65) at org.apache.solr.servlet.HttpSolrCall.writeResponse(HttpSolrCall.java:725) at org.apache.solr.servlet.HttpSolrCall.call(HttpSolrCall.java:469) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:229) at org.apache.solr.servlet.SolrDispatchFilter.doFilter(SolrDispatchFilter.java:184) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1676) at org.eclipse.jetty.servlets.CrossOriginFilter.handle(CrossOriginFilter.java:308) at org.eclipse.jetty.servlets.CrossOriginFilter.doFilter(CrossOriginFilter.java:262) at org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1668) at org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandler.java:581) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:143) at org.eclipse.jetty.security.SecurityHandler.handle(SecurityHandler.java:548) at org.eclipse.jetty.server.session.SessionHandler.doHandle(SessionHandler.java:226) at org.eclipse.jetty.server.handler.ContextHandler.doHandle(ContextHandler.java:1160) at org.eclipse.jetty.servlet.ServletHandler.doScope(ServletHandler.java:511) at org.eclipse.jetty.server.session.SessionHandler.doScope(SessionHandler.java:185) at org.eclipse.jetty.server.handler.ContextHandler.doScope(ContextHandler.java:1092) at org.eclipse.jetty.server.handler.ScopedHandler.handle(ScopedHandler.java:141) at org.eclipse.jetty.server.handler.ContextHandlerCollection.handle(ContextHandlerCollection.java:213) at org.eclipse.jetty.server.handler.HandlerCollection.handle(HandlerCollection.java:119) at org.eclipse.jetty.server.handler.HandlerWrapper.handle(HandlerWrapper.java:134) at org.eclipse.jetty.server.Server.handle(Server.java:518) at org.eclipse.jetty.server.HttpChannel.handle(HttpChannel.java:308) at org.eclipse.jetty.server.HttpConnection.onFillable(HttpConnection.java:244) at org.eclipse.jetty.io.AbstractConnection$ReadCallback.succeeded(AbstractConnection.java:273) at org.eclipse.jetty.io.FillInterest.fillable(FillInterest.java:95) at org.eclipse.jetty.io.SelectChannelEndPoint$2.run(SelectChannelEndPoint.java:93) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.produceAndRun(ExecuteProduceConsume.java:246) at org.eclipse.jetty.util.thread.strategy.ExecuteProduceConsume.run(ExecuteProduceConsume.java:156) at org.eclipse.jetty.util.thread.QueuedThreadPool.runJob(QueuedThreadPool.java:654) at org.eclipse.jetty.util.thread.QueuedThreadPool$3.run(QueuedThreadPool.java:572) at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long at org.apache.solr.client.solrj.io.stream.FacetStream.fillTuples(FacetStream.java:461) at org.apache.solr.client.solrj.io.stream.FacetStream.getTuples(FacetStream.java:420) at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:297) ... 41 more After this patch applied, ORDER BY GROUP BY clause works fine using aggregationMode=facet. $ curl --data-urlencode 'stmt=SELECT response, count(*) FROM access_log GROUP BY response' "http://localhost:8983/solr/access_log/sql?aggregationMode=facet" {"result-set":{"docs":[ {"response":200,"count(*)":1153}, {"response":404,"count(*)":3}, {"response":500,"count(*)":1}, {"EOF":true,"RESPONSE_TIME":9}]}}
        Hide
        joel.bernstein Joel Bernstein added a comment - - edited

        Ok, I'll put a test together for this. Let's make sure though that we're using the same terminology so we get the ticket name correct.

        You mention ORDER BY, but unless I'm missing something the query doesn't contain an ORDER BY. It contains a GROUP BY. If we are in agreement I'll change the name of the ticket.

        I don't believe there is a test case for grouping on a numeric field in facet mode, so I'll put a test case together for this.

        Show
        joel.bernstein Joel Bernstein added a comment - - edited Ok, I'll put a test together for this. Let's make sure though that we're using the same terminology so we get the ticket name correct. You mention ORDER BY, but unless I'm missing something the query doesn't contain an ORDER BY. It contains a GROUP BY. If we are in agreement I'll change the name of the ticket. I don't believe there is a test case for grouping on a numeric field in facet mode, so I'll put a test case together for this.
        Hide
        minoru Minoru Osuka added a comment -

        Sorry, I made a typo.
        wrong : ORDER BY
        correct: GROUP BY

        I will correct that.

        Show
        minoru Minoru Osuka added a comment - Sorry, I made a typo. wrong : ORDER BY correct: GROUP BY I will correct that.
        Hide
        joel.bernstein Joel Bernstein added a comment -

        Ok thanks for reporting this and providing a patch. I'll work up a test case for this.

        Show
        joel.bernstein Joel Bernstein added a comment - Ok thanks for reporting this and providing a patch. I'll work up a test case for this.
        Hide
        joel.bernstein Joel Bernstein added a comment - - edited

        This turning out to be difficult to reproduce. Originally I thought the issue was related to grouping on a numeric field. But after looking closely at the stack trace and the code the ClassCastException is happening when getting the count.

        Since there are testcases that exercise getting the count in facet mode already, and I've done a fair amount of manual testing that has never run into this error, I'm not sure how to reproduce the exception.

        Yonik Seeley, a Class cast exception has occurred which seems to be showing that the value from a JSON facet API count is coming back as an int. The FacetStream code treats the count value as a long, which works in the test cases. Are there scenarios where the JSON facet API would return the count as an int?

        Show
        joel.bernstein Joel Bernstein added a comment - - edited This turning out to be difficult to reproduce. Originally I thought the issue was related to grouping on a numeric field. But after looking closely at the stack trace and the code the ClassCastException is happening when getting the count . Since there are testcases that exercise getting the count in facet mode already, and I've done a fair amount of manual testing that has never run into this error, I'm not sure how to reproduce the exception. Yonik Seeley , a Class cast exception has occurred which seems to be showing that the value from a JSON facet API count is coming back as an int. The FacetStream code treats the count value as a long, which works in the test cases. Are there scenarios where the JSON facet API would return the count as an int?
        Hide
        yseeley@gmail.com Yonik Seeley added a comment -

        Are there scenarios where the JSON facet API would return the count as an int?

        Yes ("original" faceting code does the same thing).
        On a per-shard basis, "count" is normally/often returned as an int (we don't support more than that many docs per shard), but for distributed search they are summed as a long and returned that way.

        int/long are handled by the client. If you look at FacetLongMerger, it sums counts via:
        val += ((Number)facetResult).longValue();

        Show
        yseeley@gmail.com Yonik Seeley added a comment - Are there scenarios where the JSON facet API would return the count as an int? Yes ("original" faceting code does the same thing). On a per-shard basis, "count" is normally/often returned as an int (we don't support more than that many docs per shard), but for distributed search they are summed as a long and returned that way. int/long are handled by the client. If you look at FacetLongMerger, it sums counts via: val += ((Number)facetResult).longValue();
        Hide
        joel.bernstein Joel Bernstein added a comment -

        Since the FacetStream is using CloudSolrServer to make a normal distributed JSON facet call, it will in most cases be returned as a long. This explains why test cases and manual testing work with the current code. But it appears that there is some scenario, possibly a single shard scenario, where the count is returned as an int. I'll test the single shard scenario and see if this is what's happening.

        Show
        joel.bernstein Joel Bernstein added a comment - Since the FacetStream is using CloudSolrServer to make a normal distributed JSON facet call, it will in most cases be returned as a long. This explains why test cases and manual testing work with the current code. But it appears that there is some scenario, possibly a single shard scenario, where the count is returned as an int. I'll test the single shard scenario and see if this is what's happening.
        Hide
        jdyer James Dyer added a comment -

        it appears that there is some scenario, possibly a single shard scenario, where the count is returned as an int

        Indeed, if we randomize the # of shards we can reproduce this easily. See patch: SOLR-9141-test.patch .

        java.io.IOException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
        	at __randomizedtesting.SeedInfo.seed([530A362A02C855B3:F5E84F942E41E274]:0)
        	at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:351)
        ... etc ...
        	at java.lang.Thread.run(Thread.java:745)
        Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long
        	at org.apache.solr.client.solrj.io.stream.FacetStream.fillTuples(FacetStream.java:506)
        	at org.apache.solr.client.solrj.io.stream.FacetStream.getTuples(FacetStream.java:462)
        	at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:348)
        	... 41 more
        

        In fact, several of the test cases in this class fail when there is only 1 shard, but I haven't looked yet into why, or if these are valid failures.

        Show
        jdyer James Dyer added a comment - it appears that there is some scenario, possibly a single shard scenario, where the count is returned as an int Indeed, if we randomize the # of shards we can reproduce this easily. See patch: SOLR-9141-test.patch . java.io.IOException: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long at __randomizedtesting.SeedInfo.seed([530A362A02C855B3:F5E84F942E41E274]:0) at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:351) ... etc ... at java.lang.Thread.run(Thread.java:745) Caused by: java.lang.ClassCastException: java.lang.Integer cannot be cast to java.lang.Long at org.apache.solr.client.solrj.io.stream.FacetStream.fillTuples(FacetStream.java:506) at org.apache.solr.client.solrj.io.stream.FacetStream.getTuples(FacetStream.java:462) at org.apache.solr.client.solrj.io.stream.FacetStream.open(FacetStream.java:348) ... 41 more In fact, several of the test cases in this class fail when there is only 1 shard, but I haven't looked yet into why, or if these are valid failures.
        Hide
        jdyer James Dyer added a comment -

        ok, better patch, SOLR-9141-test.patch . This randomly chooses either 1, 2 or 3 shards. Parallel streams use the same # of workers as shards unless there are >2 shards, in which case a random # of workers are chosen up to the # of shards. I also made adjustments to the # of EOFs to expect, etc.

        With this, all tests in this class pass except the 2 FacetStream tests, which also pass if Minoru's patch is applied.

        I can commit both patches and resolve this issue tomorrow, unless someone objects.

        Show
        jdyer James Dyer added a comment - ok, better patch, SOLR-9141-test.patch . This randomly chooses either 1, 2 or 3 shards. Parallel streams use the same # of workers as shards unless there are >2 shards, in which case a random # of workers are chosen up to the # of shards. I also made adjustments to the # of EOFs to expect, etc. With this, all tests in this class pass except the 2 FacetStream tests, which also pass if Minoru's patch is applied. I can commit both patches and resolve this issue tomorrow, unless someone objects.
        Hide
        risdenk Kevin Risden added a comment -

        I like the addition of testing randomizing and using a different number of shards. nit but it probably makes sense to pull the numWorkers logic out into a method so it doesn't have to be adjusted in every test.

        Based on what Yonik said, for consistency could use ((Number)bucket.get("count")).longValue(); instead of the toString, cast, longValue? No real preference here.

        Show
        risdenk Kevin Risden added a comment - I like the addition of testing randomizing and using a different number of shards. nit but it probably makes sense to pull the numWorkers logic out into a method so it doesn't have to be adjusted in every test. Based on what Yonik said, for consistency could use ((Number)bucket.get("count")).longValue(); instead of the toString, cast, longValue? No real preference here.
        Hide
        joel.bernstein Joel Bernstein added a comment - - edited

        James Dyer, +1 on committing. I like the ((Number)bucket.get("count")).longValue(); approach as well.

        Show
        joel.bernstein Joel Bernstein added a comment - - edited James Dyer , +1 on committing. I like the ((Number)bucket.get("count")).longValue(); approach as well.
        Hide
        minoru Minoru Osuka added a comment -

        Joel Bernstein, +1. ((Number)bucket.get("count")).longValue(); seems faster than my patch (on my Macbook).

        package test;
        
        import java.util.ArrayList;
        import java.util.List;
        
        public class CastTest {
        	public static void main(String[] args) {
        		List<Object> objList = new ArrayList<Object>();
        		for (int i = 0; i < 10000; i++) {
        			objList.add(new Integer(i));
        		}
        
        		long start = System.nanoTime();
        		for (Object obj : objList) {
        			long l = new Long(obj.toString()).longValue();
        		}
        		long end = System.nanoTime();
        		System.out.println(String.format("new Long(obj.toString()).longValue(); : %1$,10d ns", (end - start)));
        
        		start = System.nanoTime();
        		for (Object obj : objList) {
        			long l = ((Number)obj).longValue();
        		}
        		end = System.nanoTime();
        		System.out.println(String.format("((Number)obj).longValue();            : %1$,10d ns", (end - start)));
        	}
        }
        
        new Long(obj.toString()).longValue(); : 11,301,332 ns
        ((Number)obj).longValue();            :    831,366 ns
        
        Show
        minoru Minoru Osuka added a comment - ​ Joel Bernstein , +1. ((Number)bucket.get("count")).longValue(); seems faster than my patch (on my Macbook). package test; import java.util.ArrayList; import java.util.List; public class CastTest { public static void main(String[] args) { List<Object> objList = new ArrayList<Object>(); for (int i = 0; i < 10000; i++) { objList.add(new Integer(i)); } long start = System.nanoTime(); for (Object obj : objList) { long l = new Long(obj.toString()).longValue(); } long end = System.nanoTime(); System.out.println(String.format("new Long(obj.toString()).longValue(); : %1$,10d ns", (end - start))); start = System.nanoTime(); for (Object obj : objList) { long l = ((Number)obj).longValue(); } end = System.nanoTime(); System.out.println(String.format("((Number)obj).longValue(); : %1$,10d ns", (end - start))); } } new Long(obj.toString()).longValue(); : 11,301,332 ns ((Number)obj).longValue(); : 831,366 ns
        Hide
        jdyer James Dyer added a comment -

        Here's a final patch (SOLR-9141.patch) that I will try and commit & back-port later today.

        for consistency could use ((Number)bucket.get("count")).longValue();

        of course.

        nit but it probably makes sense to pull the numWorkers logic out into a method so it doesn't have to be adjusted in every test.

        got it.

        Show
        jdyer James Dyer added a comment - Here's a final patch ( SOLR-9141.patch ) that I will try and commit & back-port later today. for consistency could use ((Number)bucket.get("count")).longValue(); of course. nit but it probably makes sense to pull the numWorkers logic out into a method so it doesn't have to be adjusted in every test. got it.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 4d4030350b79303d6f358612473f4e68570858cc in lucene-solr's branch refs/heads/master from jdyer1
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4d40303 ]

        SOLR-9141: Fix ClassCastException when using the /sql handler count() function with single-shard collections

        Show
        jira-bot ASF subversion and git services added a comment - Commit 4d4030350b79303d6f358612473f4e68570858cc in lucene-solr's branch refs/heads/master from jdyer1 [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4d40303 ] SOLR-9141 : Fix ClassCastException when using the /sql handler count() function with single-shard collections
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1609428786b17135f0d8ba413c4203b88977304b in lucene-solr's branch refs/heads/branch_6x from jdyer1
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1609428 ]

        SOLR-9141: Fix ClassCastException when using the /sql handler count() function with single-shard collections

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1609428786b17135f0d8ba413c4203b88977304b in lucene-solr's branch refs/heads/branch_6x from jdyer1 [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1609428 ] SOLR-9141 : Fix ClassCastException when using the /sql handler count() function with single-shard collections
        Hide
        jdyer James Dyer added a comment -

        Minoru, thank you for reporting this one.

        Show
        jdyer James Dyer added a comment - Minoru, thank you for reporting this one.

          People

          • Assignee:
            jdyer James Dyer
            Reporter:
            minoru Minoru Osuka
          • Votes:
            0 Vote for this issue
            Watchers:
            6 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development