Solr
  1. Solr
  2. SOLR-6547

CloudSolrServer query getqtime Exception

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 4.10, 5.3
    • Fix Version/s: 5.4, 6.0
    • Component/s: SolrJ
    • Labels:
      None

      Description

      We are using CloudSolrServer to query ,but solrj throw Exception ;
      java.lang.ClassCastException: java.lang.Long cannot be cast to java.lang.Integer at org.apache.solr.client.solrj.response.SolrResponseBase.getQTime(SolrResponseBase.java:76)

      1. SOLR-6547.patch
        5 kB
        Shalin Shekhar Mangar
      2. SOLR-6547.patch
        0.8 kB
        Noble Paul
      3. SOLR-6547.patch
        1.0 kB
        Anurag Sharma

        Activity

        Hide
        Anurag Sharma added a comment -

        Can you mention the exact query it will become easy to investigate/reproduce

        Show
        Anurag Sharma added a comment - Can you mention the exact query it will become easy to investigate/reproduce
        Hide
        Hoss Man added a comment -

        i suspect this has something to do with the code in CloudSolrServer.condenseResponse ?

        I honestly have no idea what that method is for or what it's doing, but a quick grep suggests that's the only place in the code base where a "long" value is getting assocaited with "QTime"

        • CloudSolrServer.condenseResponse should be fixed to use an int for QTime
        • SolrResponseBase.getQTime can (and probably should) be hardended to do something like this instead of just a direct Integer cast...
          return ((Number) header.get("QTime")).intValue()
          

          ...that kind of pattern can help protect us in the future if we decide that QTime should be a long

        Show
        Hoss Man added a comment - i suspect this has something to do with the code in CloudSolrServer.condenseResponse ? I honestly have no idea what that method is for or what it's doing, but a quick grep suggests that's the only place in the code base where a "long" value is getting assocaited with "QTime" CloudSolrServer.condenseResponse should be fixed to use an int for QTime SolrResponseBase.getQTime can (and probably should) be hardended to do something like this instead of just a direct Integer cast... return (( Number ) header.get( "QTime" )).intValue() ...that kind of pattern can help protect us in the future if we decide that QTime should be a long
        Hide
        Anurag Sharma added a comment -

        Hi Hoss,

        Thanks for your update.

        Time is usually returned as long value. Since getQTime returns, int looks like the interest is to return value in seconds and not milliseconds.

        I agree with your suggestion to change the return type to either long or use intValue while returning in the current method.

        Is there a way I can reproduce the java.lang.ClassCastException mentioned in the description?
        Kevin - Any suggestion?

        Anurag

        Show
        Anurag Sharma added a comment - Hi Hoss, Thanks for your update. Time is usually returned as long value. Since getQTime returns, int looks like the interest is to return value in seconds and not milliseconds. I agree with your suggestion to change the return type to either long or use intValue while returning in the current method. Is there a way I can reproduce the java.lang.ClassCastException mentioned in the description? Kevin - Any suggestion? Anurag
        Hide
        Shawn Heisey added a comment -

        There's merit either way.

        Integer.MAX_VALUE in milliseconds is a little more than 24 days. I doubt you would ever run into a query that would actually take that long and still have enough resources to actually finish, so an int has plenty of capacity to handle QTime. The difference is tiny for a single value, but a long does take up twice as much memory. If most of the existing code uses an int, I am inclined to stick with that and change the deviating code.

        On the other hand ... everything else related to time in Java is represented as a long, as Anurag Sharma mentioned. If the value is ever used for calculation with any of those other numbers, it might save a tiny bit of CPU time if Java doesn't have to convert.

        Switching everything to a long is likely to involve quite a bit of effort to locate and change every place that it's used, which is a big part of why I suggest sticking with int. I don't think we would ever need a long for relative times.

        Show
        Shawn Heisey added a comment - There's merit either way. Integer.MAX_VALUE in milliseconds is a little more than 24 days. I doubt you would ever run into a query that would actually take that long and still have enough resources to actually finish, so an int has plenty of capacity to handle QTime. The difference is tiny for a single value, but a long does take up twice as much memory. If most of the existing code uses an int, I am inclined to stick with that and change the deviating code. On the other hand ... everything else related to time in Java is represented as a long, as Anurag Sharma mentioned. If the value is ever used for calculation with any of those other numbers, it might save a tiny bit of CPU time if Java doesn't have to convert. Switching everything to a long is likely to involve quite a bit of effort to locate and change every place that it's used, which is a big part of why I suggest sticking with int. I don't think we would ever need a long for relative times.
        Hide
        Anurag Sharma added a comment -

        I am not sure of the usage. If there is a use case to calculate the response time then long would be precise to respond the query time in millisecond. With int the millisecond detail is missed.

        Show
        Anurag Sharma added a comment - I am not sure of the usage. If there is a use case to calculate the response time then long would be precise to respond the query time in millisecond. With int the millisecond detail is missed.
        Hide
        Shawn Heisey added a comment - - edited

        QTime is always in milliseconds, no matter what size the primitive datatype is. The maximum value of a signed integer is 2147483647. In seconds, this is about 68 years – the UNIX epoch is the beginning of 1970, and the 32-bit signed counter will run out in 2038. In milliseconds, this number is only 24 days.

        Show
        Shawn Heisey added a comment - - edited QTime is always in milliseconds, no matter what size the primitive datatype is. The maximum value of a signed integer is 2147483647. In seconds, this is about 68 years – the UNIX epoch is the beginning of 1970, and the 32-bit signed counter will run out in 2038. In milliseconds, this number is only 24 days.
        Hide
        Anurag Sharma added a comment -

        Based on the above comment we are good to go with Hoss Mann comment to extract the intValue from long:

        return ((Number) header.get("QTime")).intValue()
        

        A couple of failing unit tests can help in fixing and creating the patch.

        Show
        Anurag Sharma added a comment - Based on the above comment we are good to go with Hoss Mann comment to extract the intValue from long: return (( Number ) header.get( "QTime" )).intValue() A couple of failing unit tests can help in fixing and creating the patch.
        Hide
        Anurag Sharma added a comment -

        QTime is saved as long after truncating nano second part of time. This is equivalent to capturing the time in milliseconds. Here is the code snippet of what's happening in the CloudSolrServer.

        long start = System.nanoTime();
        .
        .
        long end = System.nanoTime();
        RouteResponse rr =  condenseResponse(shardResponses, (long)((end - start)/1000000));
        

        condenseResponse function

        public RouteResponse condenseResponse(NamedList response, long timeMillis) {
          .
          .
          cheader.add("QTime", timeMillis);
          .
          .
        }
        

        Since the time in seconds can be captured with Integer, there are two ways to fix the issue:

        1. In CloudSolrServer, truncate the milliseconds part as well and save QTime in Integer. This way getQTime won't throw Long to Integer ClassCastException as the object coming to it already Integer.
        2. In SolrResponseBase.getQTime function, check the instanceOf Object and get integer from it as shown in the code snippet below
                  int qtime = 0;
                  if(obj instanceof Long) {
                  	qtime = (int)(((Long) obj).longValue()/1000);
                  } else if (obj instanceof Integer) {
                  	qtime = (Integer)obj;
                  } else if (obj instanceof String) {
                  	qtime = Integer.parseInt((String) obj);        	
                  }
                  return qtime;
          

        Please vote for proceeding the best approach. Also like to get opinion on writing unit test in CloudSolrServerTest class.

        Show
        Anurag Sharma added a comment - QTime is saved as long after truncating nano second part of time. This is equivalent to capturing the time in milliseconds. Here is the code snippet of what's happening in the CloudSolrServer. long start = System .nanoTime(); . . long end = System .nanoTime(); RouteResponse rr = condenseResponse(shardResponses, ( long )((end - start)/1000000)); condenseResponse function public RouteResponse condenseResponse(NamedList response, long timeMillis) { . . cheader.add( "QTime" , timeMillis); . . } Since the time in seconds can be captured with Integer, there are two ways to fix the issue: In CloudSolrServer, truncate the milliseconds part as well and save QTime in Integer. This way getQTime won't throw Long to Integer ClassCastException as the object coming to it already Integer. In SolrResponseBase.getQTime function, check the instanceOf Object and get integer from it as shown in the code snippet below int qtime = 0; if (obj instanceof Long ) { qtime = ( int )((( Long ) obj).longValue()/1000); } else if (obj instanceof Integer ) { qtime = ( Integer )obj; } else if (obj instanceof String ) { qtime = Integer .parseInt(( String ) obj); } return qtime; Please vote for proceeding the best approach. Also like to get opinion on writing unit test in CloudSolrServerTest class.
        Hide
        Anurag Sharma added a comment -

        Fix using #2 approach, without UT.

        Show
        Anurag Sharma added a comment - Fix using #2 approach, without UT.
        Hide
        Noble Paul added a comment -

        no need to check for string etc.

        Is it possible to get a testcase for this?

        Show
        Noble Paul added a comment - no need to check for string etc. Is it possible to get a testcase for this?
        Hide
        Shalin Shekhar Mangar added a comment -

        Test + fix. I'll commit shortly.

        Show
        Shalin Shekhar Mangar added a comment - Test + fix. I'll commit shortly.
        Hide
        ASF subversion and git services added a comment -

        Commit 1703335 from shalin@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1703335 ]

        SOLR-6547: ClassCastException in SolrResponseBase.getQTime

        Show
        ASF subversion and git services added a comment - Commit 1703335 from shalin@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1703335 ] SOLR-6547 : ClassCastException in SolrResponseBase.getQTime
        Hide
        ASF subversion and git services added a comment -

        Commit 1703343 from shalin@apache.org in branch 'dev/branches/branch_5x'
        [ https://svn.apache.org/r1703343 ]

        SOLR-6547: ClassCastException in SolrResponseBase.getQTime

        Show
        ASF subversion and git services added a comment - Commit 1703343 from shalin@apache.org in branch 'dev/branches/branch_5x' [ https://svn.apache.org/r1703343 ] SOLR-6547 : ClassCastException in SolrResponseBase.getQTime

          People

          • Assignee:
            Shalin Shekhar Mangar
            Reporter:
            kevin
          • Votes:
            1 Vote for this issue
            Watchers:
            8 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development