Solr
  1. Solr
  2. SOLR-6037

Stats Component Reports Incorrect max/sum/stddev for date fields in Distributed Data

    Details

      Description

      When computing statistics using the statistics component, the value for max may be reported incorrectly if the maximum value does not lie in the first shard.

      This is where the problem is:

      StatsValuesFactory.java
        @Override
        protected void updateMinMax(Date min, Date max) {
          if(this.min==null || this.min.after(min)) {
            this.min = min;
          }
          if(this.max==null || this.max.before(min)) {
            this.max = max;
          }
        }
      

      Attached is a one-line patch which fixes this issue.

      1. SOLR-6037.patch
        3 kB
        Hoss Man
      2. SOLR-6037.patch
        2 kB
        Hoss Man
      3. SOLR-6037.patch
        1 kB
        Brett Lucey

        Activity

        Hide
        Hoss Man added a comment -

        Ugh...

        Brett: thanks for reporting this.

        I've updated your patch to account for the fact that min/max might be null coming back from a shard. When updating the existing distributed stats test to also include date fields to trigger this problem, I uncovered another glitch in how the sum/stddev are computed (sum was inited to "-1" for some bizare reason) which is also fixed in this updated patch.

        Show
        Hoss Man added a comment - Ugh... Brett: thanks for reporting this. I've updated your patch to account for the fact that min/max might be null coming back from a shard. When updating the existing distributed stats test to also include date fields to trigger this problem, I uncovered another glitch in how the sum/stddev are computed (sum was inited to "-1" for some bizare reason) which is also fixed in this updated patch.
        Hide
        Hoss Man added a comment -

        updating summary

        Show
        Hoss Man added a comment - updating summary
        Hide
        Hoss Man added a comment -

        updated patch with a bit more test coverage

        Show
        Hoss Man added a comment - updated patch with a bit more test coverage
        Hide
        ASF subversion and git services added a comment -

        Commit 1591800 from hossman@apache.org in branch 'dev/trunk'
        [ https://svn.apache.org/r1591800 ]

        SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent

        Show
        ASF subversion and git services added a comment - Commit 1591800 from hossman@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1591800 ] SOLR-6037 : Fixed incorrect max/sum/stddev for Date fields in StatsComponent
        Hide
        ASF subversion and git services added a comment -

        Commit 1591801 from hossman@apache.org in branch 'dev/branches/branch_4x'
        [ https://svn.apache.org/r1591801 ]

        SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent (merge r1591800)

        Show
        ASF subversion and git services added a comment - Commit 1591801 from hossman@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591801 ] SOLR-6037 : Fixed incorrect max/sum/stddev for Date fields in StatsComponent (merge r1591800)
        Hide
        ASF subversion and git services added a comment -

        Commit 1592821 from hossman@apache.org in branch 'dev/branches/lucene_solr_4_8'
        [ https://svn.apache.org/r1592821 ]

        Merge back several "low hanging" bug fixes into the 4.8 branch...
        SOLR-6029: r1590867
        SOLR-6030: r1591555
        SOLR-6037: r1591800
        SOLR-6023: r1592195
        SOLR-5090: r1592591
        SOLR-6039: r1592605
        SOLR-5993: r1588415
        SOLR-5904: r1587702

        Show
        ASF subversion and git services added a comment - Commit 1592821 from hossman@apache.org in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1592821 ] Merge back several "low hanging" bug fixes into the 4.8 branch... SOLR-6029 : r1590867 SOLR-6030 : r1591555 SOLR-6037 : r1591800 SOLR-6023 : r1592195 SOLR-5090 : r1592591 SOLR-6039 : r1592605 SOLR-5993 : r1588415 SOLR-5904 : r1587702

          People

          • Assignee:
            Hoss Man
            Reporter:
            Brett Lucey
          • Votes:
            4 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development