Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7291

HeatmapFacetCounter bug with dateline and large non-point shapes

    Details

    • Lucene Fields:
      New

      Description

      Jenkins found a test failure today.
      This reproduces for me (master, java 8):
      ant test -Dtestcase=HeatmapFacetCounterTest -Dtests.method=testRandom -Dtests.seed=3EC907D1784B6F23 -Dtests.multiplier=2 -Dtests.nightly=true -Dtests.slow=true -Dtests.linedocsfile=/x1/jenkins/lucene-data/enwiki.random.lines.txt -Dtests.locale=is-IS -Dtests.timezone=Europe/Tirane -Dtests.asserts=true -Dtests.file.encoding=UTF-8

      java.lang.AssertionError: 
      Expected :1
      Actual   :0
       <Click to see difference>
      
      
      	at __randomizedtesting.SeedInfo.seed([3EC907D1784B6F23:A3439C5F68FEAB94]:0)
      	at org.junit.Assert.fail(Assert.java:93)
      	at org.junit.Assert.failNotEquals(Assert.java:647)
      	at org.junit.Assert.assertEquals(Assert.java:128)
      	at org.junit.Assert.assertEquals(Assert.java:472)
      	at org.junit.Assert.assertEquals(Assert.java:456)
      	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.validateHeatmapResult(HeatmapFacetCounterTest.java:226)
      	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:193)
      	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.queryHeatmapRecursive(HeatmapFacetCounterTest.java:206)
      	at org.apache.lucene.spatial.prefix.HeatmapFacetCounterTest.testRandom(HeatmapFacetCounterTest.java:172)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner.invoke(RandomizedRunner.java:1764)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$8.evaluate(RandomizedRunner.java:871)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$9.evaluate(RandomizedRunner.java:907)
      	at com.carrotsearch.randomizedtesting.RandomizedRunner$10.evaluate(RandomizedRunner.java:921)
      

        Activity

        Hide
        jpountz Adrien Grand added a comment -

        David Smiley Pinging you in case you want to have a chance to look into it before we release 6.1. FYI the seed still reproduces for me on master.

        Show
        jpountz Adrien Grand added a comment - David Smiley Pinging you in case you want to have a chance to look into it before we release 6.1. FYI the seed still reproduces for me on master.
        Hide
        dsmiley David Smiley added a comment -

        This is a real bug; the patch I'm attaching fixes it and adds a test for the input that reproduced this circumstance.

        A faulty count accumulation can happen in this circumstance:

        • indexed data includes non-point shapes
        • heatmap region crosses the dateline
        • one of the indexed shapes is a good deal larger than the heatmap region such that it has leaf cells larger than the target heatmap cell size (so-called "ancestors" in the code).

        I'll commit this to master, 6x, and 6.1

        Show
        dsmiley David Smiley added a comment - This is a real bug; the patch I'm attaching fixes it and adds a test for the input that reproduced this circumstance. A faulty count accumulation can happen in this circumstance: indexed data includes non-point shapes heatmap region crosses the dateline one of the indexed shapes is a good deal larger than the heatmap region such that it has leaf cells larger than the target heatmap cell size (so-called "ancestors" in the code). I'll commit this to master, 6x, and 6.1
        Hide
        jpountz Adrien Grand added a comment -

        +1

        Did you comment out the "@Repeat(iterations = 20)" on purpose?

        Show
        jpountz Adrien Grand added a comment - +1 Did you comment out the "@Repeat(iterations = 20)" on purpose?
        Hide
        dsmiley David Smiley added a comment -

        No I didn't; I already caught that and will commit that unchanged.

        Show
        dsmiley David Smiley added a comment - No I didn't; I already caught that and will commit that unchanged.
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit b33d7176aa3624df2de1708b17919f20d034872f in lucene-solr's branch refs/heads/master from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b33d717 ]

        LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes

        Show
        jira-bot ASF subversion and git services added a comment - Commit b33d7176aa3624df2de1708b17919f20d034872f in lucene-solr's branch refs/heads/master from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=b33d717 ] LUCENE-7291 : Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 7520d79e040c16c5ab666f1ad28c8665fb0ceb40 in lucene-solr's branch refs/heads/branch_6x from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7520d79 ]

        LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes
        (cherry picked from commit b33d717)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 7520d79e040c16c5ab666f1ad28c8665fb0ceb40 in lucene-solr's branch refs/heads/branch_6x from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=7520d79 ] LUCENE-7291 : Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes (cherry picked from commit b33d717)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 6372c0b4042ec2c8d94e50e5c2b9c1df469414e2 in lucene-solr's branch refs/heads/branch_6_1 from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6372c0b ]

        LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes
        (cherry picked from commit 7520d79)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 6372c0b4042ec2c8d94e50e5c2b9c1df469414e2 in lucene-solr's branch refs/heads/branch_6_1 from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6372c0b ] LUCENE-7291 : Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes (cherry picked from commit 7520d79)
        Hide
        steve_rowe Steve Rowe added a comment -

        Reopening to backport to 6.0.2, 5.6 and 5.5.2

        Show
        steve_rowe Steve Rowe added a comment - Reopening to backport to 6.0.2, 5.6 and 5.5.2
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 5c546537d7b8130c05263832baff4946260f6a31 in lucene-solr's branch refs/heads/branch_5_5 from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5c54653 ]

        LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes
        (cherry picked from commit 7520d79)

        Show
        jira-bot ASF subversion and git services added a comment - Commit 5c546537d7b8130c05263832baff4946260f6a31 in lucene-solr's branch refs/heads/branch_5_5 from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=5c54653 ] LUCENE-7291 : Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes (cherry picked from commit 7520d79)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit 1d7ad90947699e103de39fded5b78f76a30e449b in lucene-solr's branch refs/heads/branch_5_5 from Steve Rowe
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1d7ad90 ]

        LUCENE-7291: Add 5.5.2 CHANGES entry

        Show
        jira-bot ASF subversion and git services added a comment - Commit 1d7ad90947699e103de39fded5b78f76a30e449b in lucene-solr's branch refs/heads/branch_5_5 from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1d7ad90 ] LUCENE-7291 : Add 5.5.2 CHANGES entry
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit f6b0fb95dea43f9f508b613cf32f489aaa263c4e in lucene-solr's branch refs/heads/branch_5x from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f6b0fb9 ]

        LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes
        (cherry picked from commit 7520d79)

        Show
        jira-bot ASF subversion and git services added a comment - Commit f6b0fb95dea43f9f508b613cf32f489aaa263c4e in lucene-solr's branch refs/heads/branch_5x from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=f6b0fb9 ] LUCENE-7291 : Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes (cherry picked from commit 7520d79)
        Hide
        jira-bot ASF subversion and git services added a comment -

        Commit a7f2876ec5ce9ca5ef271cad97027a5cb5e43619 in lucene-solr's branch refs/heads/branch_6_0 from David Smiley
        [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a7f2876 ]

        LUCENE-7291: Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes
        (cherry picked from commit 7520d79)

        Show
        jira-bot ASF subversion and git services added a comment - Commit a7f2876ec5ce9ca5ef271cad97027a5cb5e43619 in lucene-solr's branch refs/heads/branch_6_0 from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=a7f2876 ] LUCENE-7291 : Fix spatial HeatmapFacetCounter bug with dateline and large non-point shapes (cherry picked from commit 7520d79)

          People

          • Assignee:
            dsmiley David Smiley
            Reporter:
            dsmiley David Smiley
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development