Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-10082

Add Variance and Standard Deviation aggregators

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 7.0
    • Fix Version/s: 6.6, 7.0
    • Component/s: faceting
    • Security Level: Public (Default Security Level. Issues are Public)
    • Labels:
      None

      Description

      This is a ticket to add variance and standard deviation aggregators to json facet.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user rustamhsmv opened a pull request:

          https://github.com/apache/lucene-solr/pull/155

          SOLR-10082 : Add Variance and Standard Deviation aggregators

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/rustamhsmv/lucene-solr VarianceStddev

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/lucene-solr/pull/155.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #155


          commit 6e1e1ac0296400bd87def4fa62ebab9541c1f9a4
          Author: rustamhsmv <rustam.hashimov@outlook.com>
          Date: 2017-01-31T22:34:05Z

          Add Variance and Stddev Aggregators

          commit 672138bf34bb7eee43add03e5d04aaff25955d17
          Author: rustamhsmv <rustam.hashimov@outlook.com>
          Date: 2017-01-31T22:40:20Z

          Merge branch 'master' of https://github.com/rustamhsmv/lucene-solr into VarianceStddev


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user rustamhsmv opened a pull request: https://github.com/apache/lucene-solr/pull/155 SOLR-10082 : Add Variance and Standard Deviation aggregators You can merge this pull request into a Git repository by running: $ git pull https://github.com/rustamhsmv/lucene-solr VarianceStddev Alternatively you can review and apply these changes as the patch at: https://github.com/apache/lucene-solr/pull/155.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #155 commit 6e1e1ac0296400bd87def4fa62ebab9541c1f9a4 Author: rustamhsmv <rustam.hashimov@outlook.com> Date: 2017-01-31T22:34:05Z Add Variance and Stddev Aggregators commit 672138bf34bb7eee43add03e5d04aaff25955d17 Author: rustamhsmv <rustam.hashimov@outlook.com> Date: 2017-01-31T22:40:20Z Merge branch 'master' of https://github.com/rustamhsmv/lucene-solr into VarianceStddev
          Hide
          yseeley@gmail.com Yonik Seeley added a comment - - edited

          I'm taking a look at this now...
          Regarding the rounding up of variance and standard deviation to the nearest 10th... I'm inclined to remove it since it would seem to be dependent on the domain?

          further, it doesn't look like the code would actually round up as the comments describe:

          return Math.round(val * 10) / 10.0; // round up to 1 decimal place
          
          Show
          yseeley@gmail.com Yonik Seeley added a comment - - edited I'm taking a look at this now... Regarding the rounding up of variance and standard deviation to the nearest 10th... I'm inclined to remove it since it would seem to be dependent on the domain? further, it doesn't look like the code would actually round up as the comments describe: return Math .round(val * 10) / 10.0; // round up to 1 decimal place
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          After removing the rounding, I'm seeing much larger variations than I would expect for distrib vs non-distrib. As an example:
          '74.7'!='74.66666666666667'

          At this point, I'm not sure if the numerical methods used here are that unstable (squaring big things can tend to swamp smaller contributions), or if there is another bug somewhere that the rounding was obscuring.

          Show
          yseeley@gmail.com Yonik Seeley added a comment - After removing the rounding, I'm seeing much larger variations than I would expect for distrib vs non-distrib. As an example: '74.7'!='74.66666666666667' At this point, I'm not sure if the numerical methods used here are that unstable (squaring big things can tend to swamp smaller contributions), or if there is another bug somewhere that the rounding was obscuring.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 3145f781b3b55ebbd6374f40db7fe7b676b8d1af in lucene-solr's branch refs/heads/master from Yonik Seeley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3145f78 ]

          SOLR-10082: JSON Facet API, add stddev and variance functions

          Show
          jira-bot ASF subversion and git services added a comment - Commit 3145f781b3b55ebbd6374f40db7fe7b676b8d1af in lucene-solr's branch refs/heads/master from Yonik Seeley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3145f78 ] SOLR-10082 : JSON Facet API, add stddev and variance functions
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 46012784d4999790190a8d6c3d8cc697f7541160 in lucene-solr's branch refs/heads/branch_6x from Yonik Seeley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4601278 ]

          SOLR-10082: JSON Facet API, add stddev and variance functions

          Show
          jira-bot ASF subversion and git services added a comment - Commit 46012784d4999790190a8d6c3d8cc697f7541160 in lucene-solr's branch refs/heads/branch_6x from Yonik Seeley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=4601278 ] SOLR-10082 : JSON Facet API, add stddev and variance functions
          Hide
          yseeley@gmail.com Yonik Seeley added a comment -

          OK, I found the issues (I missed some places that were still rounding) and committed. Thanks Rustam!

          Show
          yseeley@gmail.com Yonik Seeley added a comment - OK, I found the issues (I missed some places that were still rounding) and committed. Thanks Rustam!

            People

            • Assignee:
              yseeley@gmail.com Yonik Seeley
              Reporter:
              rustamhsmv Rustam Hashimov
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development