Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-6037

Stats Component Reports Incorrect max/sum/stddev for date fields in Distributed Data

Details

    Description

      When computing statistics using the statistics component, the value for max may be reported incorrectly if the maximum value does not lie in the first shard.

      This is where the problem is:

      StatsValuesFactory.java
        @Override
        protected void updateMinMax(Date min, Date max) {
          if(this.min==null || this.min.after(min)) {
            this.min = min;
          }
          if(this.max==null || this.max.before(min)) {
            this.max = max;
          }
        }
      

      Attached is a one-line patch which fixes this issue.

      Attachments

        1. SOLR-6037.patch
          1 kB
          Brett Lucey
        2. SOLR-6037.patch
          2 kB
          Chris M. Hostetter
        3. SOLR-6037.patch
          3 kB
          Chris M. Hostetter

        Activity

          Ugh...

          Brett: thanks for reporting this.

          I've updated your patch to account for the fact that min/max might be null coming back from a shard. When updating the existing distributed stats test to also include date fields to trigger this problem, I uncovered another glitch in how the sum/stddev are computed (sum was inited to "-1" for some bizare reason) which is also fixed in this updated patch.

          hossman Chris M. Hostetter added a comment - Ugh... Brett: thanks for reporting this. I've updated your patch to account for the fact that min/max might be null coming back from a shard. When updating the existing distributed stats test to also include date fields to trigger this problem, I uncovered another glitch in how the sum/stddev are computed (sum was inited to "-1" for some bizare reason) which is also fixed in this updated patch.

          updating summary

          hossman Chris M. Hostetter added a comment - updating summary

          updated patch with a bit more test coverage

          hossman Chris M. Hostetter added a comment - updated patch with a bit more test coverage

          Commit 1591800 from hossman@apache.org in branch 'dev/trunk'
          [ https://svn.apache.org/r1591800 ]

          SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent

          jira-bot ASF subversion and git services added a comment - Commit 1591800 from hossman@apache.org in branch 'dev/trunk' [ https://svn.apache.org/r1591800 ] SOLR-6037 : Fixed incorrect max/sum/stddev for Date fields in StatsComponent

          Commit 1591801 from hossman@apache.org in branch 'dev/branches/branch_4x'
          [ https://svn.apache.org/r1591801 ]

          SOLR-6037: Fixed incorrect max/sum/stddev for Date fields in StatsComponent (merge r1591800)

          jira-bot ASF subversion and git services added a comment - Commit 1591801 from hossman@apache.org in branch 'dev/branches/branch_4x' [ https://svn.apache.org/r1591801 ] SOLR-6037 : Fixed incorrect max/sum/stddev for Date fields in StatsComponent (merge r1591800)

          Commit 1592821 from hossman@apache.org in branch 'dev/branches/lucene_solr_4_8'
          [ https://svn.apache.org/r1592821 ]

          Merge back several "low hanging" bug fixes into the 4.8 branch...
          SOLR-6029: r1590867
          SOLR-6030: r1591555
          SOLR-6037: r1591800
          SOLR-6023: r1592195
          SOLR-5090: r1592591
          SOLR-6039: r1592605
          SOLR-5993: r1588415
          SOLR-5904: r1587702

          jira-bot ASF subversion and git services added a comment - Commit 1592821 from hossman@apache.org in branch 'dev/branches/lucene_solr_4_8' [ https://svn.apache.org/r1592821 ] Merge back several "low hanging" bug fixes into the 4.8 branch... SOLR-6029 : r1590867 SOLR-6030 : r1591555 SOLR-6037 : r1591800 SOLR-6023 : r1592195 SOLR-5090 : r1592591 SOLR-6039 : r1592605 SOLR-5993 : r1588415 SOLR-5904 : r1587702

          People

            hossman Chris M. Hostetter
            brett.lucey Brett Lucey
            Votes:
            4 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: