Uploaded image for project: 'Lucene - Core'
  1. Lucene - Core
  2. LUCENE-7278

Make template Calendar configurable in DateRangePrefixTree

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 6.1
    • Component/s: modules/spatial-extras
    • Labels:
      None
    • Lucene Fields:
      New

      Description

      DateRangePrefixTree (a SpatialPrefixTree designed for dates and date ranges) currently uses a hard-coded Calendar template for making new instances. This ought to be configurable so that, for example, the Gregorian change date can be configured. This is particularly important for compatibility with Java 8's java.time API which uses the Gregorian calendar for all time (there is no use of Julian prior to 1582).

      1. LUCENE_7278.patch
        20 kB
        David Smiley
      2. LUCENE_7278.patch
        19 kB
        David Smiley
      3. LUCENE-7278-clover.patch
        1 kB
        Steve Rowe

        Issue Links

          Activity

          Hide
          dsmiley David Smiley added a comment -

          The attached patch deprecates the INSTANCE singleton and makes the constructor public to accept a Calendar. 2 static instances are provided, one based on the default Calendar.getInstance() and the other with no GCD set.

          I also improved toString(Calendar) to not use SimpleDateFormatter any more (yay!). Also, it now has format parity with java.time.DateTimeFormatter.ISO_INSTANT which means using a leading '+' beyond 9999; though still no trailing 'Z'.

          Show
          dsmiley David Smiley added a comment - The attached patch deprecates the INSTANCE singleton and makes the constructor public to accept a Calendar. 2 static instances are provided, one based on the default Calendar.getInstance() and the other with no GCD set. I also improved toString(Calendar) to not use SimpleDateFormatter any more (yay!). Also, it now has format parity with java.time.DateTimeFormatter.ISO_INSTANT which means using a leading '+' beyond 9999; though still no trailing 'Z'.
          Hide
          dsmiley David Smiley added a comment -

          Small bug fixed: the toString(cal) should not emit the milliseconds if it's 0. The randomized test would eventually find it (1/1000) but I caught this at a test using this in Solr.

          Show
          dsmiley David Smiley added a comment - Small bug fixed: the toString(cal) should not emit the milliseconds if it's 0. The randomized test would eventually find it (1/1000) but I caught this at a test using this in Solr.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 062869626f321ee3702bc0a48ff34199f397fd70 in lucene-solr's branch refs/heads/master from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0628696 ]

          LUCENE-7278: DateRangePrefixTree's Calendar is now configurable

          Show
          jira-bot ASF subversion and git services added a comment - Commit 062869626f321ee3702bc0a48ff34199f397fd70 in lucene-solr's branch refs/heads/master from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=0628696 ] LUCENE-7278 : DateRangePrefixTree's Calendar is now configurable
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1f6487a7cc680a38429e156aa85a48f06e73378b in lucene-solr's branch refs/heads/branch_6x from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1f6487a ]

          LUCENE-7278: DateRangePrefixTree's Calendar is now configurable
          (cherry picked from commit 0628696)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1f6487a7cc680a38429e156aa85a48f06e73378b in lucene-solr's branch refs/heads/branch_6x from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1f6487a ] LUCENE-7278 : DateRangePrefixTree's Calendar is now configurable (cherry picked from commit 0628696)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 1d35bd0ea8beb635997c3cf131ded8ebe58d15a9 in lucene-solr's branch refs/heads/master from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1d35bd0 ]

          LUCENE-7278: DRPT: fix bug in assert statement

          Show
          jira-bot ASF subversion and git services added a comment - Commit 1d35bd0ea8beb635997c3cf131ded8ebe58d15a9 in lucene-solr's branch refs/heads/master from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=1d35bd0 ] LUCENE-7278 : DRPT: fix bug in assert statement
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 32e16cb0e60cba8057d47be45ab022ba92e254ad in lucene-solr's branch refs/heads/branch_6x from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=32e16cb ]

          LUCENE-7278: DRPT: fix bug in assert statement
          (cherry picked from commit 1d35bd0)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 32e16cb0e60cba8057d47be45ab022ba92e254ad in lucene-solr's branch refs/heads/branch_6x from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=32e16cb ] LUCENE-7278 : DRPT: fix bug in assert statement (cherry picked from commit 1d35bd0)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit af39ab8004927e88aa37e015ee92e752c0a12163 in lucene-solr's branch refs/heads/branch_6_0 from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=af39ab8 ]

          LUCENE-7278: DateRangePrefixTree's Calendar is now configurable.
          (cherry picked from commit 1f6487a and 32e16cb0e60cba8057d47be45ab022ba92e254ad)

          Show
          jira-bot ASF subversion and git services added a comment - Commit af39ab8004927e88aa37e015ee92e752c0a12163 in lucene-solr's branch refs/heads/branch_6_0 from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=af39ab8 ] LUCENE-7278 : DateRangePrefixTree's Calendar is now configurable. (cherry picked from commit 1f6487a and 32e16cb0e60cba8057d47be45ab022ba92e254ad)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 2accf12d710f743b51bbc24f613a36f51b572e37 in lucene-solr's branch refs/heads/master from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2accf12 ]

          LUCENE-7278: DRPT: fix test bug (when milli is 0)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 2accf12d710f743b51bbc24f613a36f51b572e37 in lucene-solr's branch refs/heads/master from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=2accf12 ] LUCENE-7278 : DRPT: fix test bug (when milli is 0)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit bd3e568754ac0b4b96e4a955387c413e0770e871 in lucene-solr's branch refs/heads/branch_6x from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bd3e568 ]

          LUCENE-7278: DRPT: fix test bug (when milli is 0)
          (cherry picked from commit 2accf12)

          Show
          jira-bot ASF subversion and git services added a comment - Commit bd3e568754ac0b4b96e4a955387c413e0770e871 in lucene-solr's branch refs/heads/branch_6x from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=bd3e568 ] LUCENE-7278 : DRPT: fix test bug (when milli is 0) (cherry picked from commit 2accf12)
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 14af6994ea2734e91616f7f23ed90c7b4f050018 in lucene-solr's branch refs/heads/branch_6_0 from David Smiley
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=14af699 ]

          LUCENE-7278: DRPT: fix test bug (when milli is 0)
          (cherry picked from commit bd3e568)

          Show
          jira-bot ASF subversion and git services added a comment - Commit 14af6994ea2734e91616f7f23ed90c7b4f050018 in lucene-solr's branch refs/heads/branch_6_0 from David Smiley [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=14af699 ] LUCENE-7278 : DRPT: fix test bug (when milli is 0) (cherry picked from commit bd3e568)
          Hide
          steve_rowe Steve Rowe added a comment -

          Clover has been failing on ASF Jenkins since this was committed, e.g. from https://builds.apache.org/job/Lucene-Solr-Clover-master/438/consoleText:

          Caused by: java.io.FileNotFoundException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Clover-master/lucene/build/clover/reports/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest_testToStringISO8601__p0_java_util_GregorianCalendar_time___areFieldsSet_false_areAllFieldsSet_false_lenient_true_zone_sun_util_calendar_ZoneInfo_id__UTC__offset_0_dstSavings_0_useDaylight_false_transitions_0_lastRule_null__firstDayOfWeek_2_minimalDaysInFirstWeek_4_ERA___YEAR___MONTH___WEEK_OF_YEAR___WEEK_OF_MONTH___DAY_OF_MONTH___DAY_OF_YEAR___DAY_OF_WEEK___DAY_OF_WEEK_IN_MONTH___AM_PM___HOUR___HOUR_OF_DAY___MINUTE___SECOND___MILLISECOND___ZONE_OFFSET___DST_OFFSET_____-535x98.html (File name too long)
          
          Show
          steve_rowe Steve Rowe added a comment - Clover has been failing on ASF Jenkins since this was committed, e.g. from https://builds.apache.org/job/Lucene-Solr-Clover-master/438/consoleText : Caused by: java.io.FileNotFoundException: /x1/jenkins/jenkins-slave/workspace/Lucene-Solr-Clover-master/lucene/build/clover/reports/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest_testToStringISO8601__p0_java_util_GregorianCalendar_time___areFieldsSet_false_areAllFieldsSet_false_lenient_true_zone_sun_util_calendar_ZoneInfo_id__UTC__offset_0_dstSavings_0_useDaylight_false_transitions_0_lastRule_null__firstDayOfWeek_2_minimalDaysInFirstWeek_4_ERA___YEAR___MONTH___WEEK_OF_YEAR___WEEK_OF_MONTH___DAY_OF_MONTH___DAY_OF_YEAR___DAY_OF_WEEK___DAY_OF_WEEK_IN_MONTH___AM_PM___HOUR___HOUR_OF_DAY___MINUTE___SECOND___MILLISECOND___ZONE_OFFSET___DST_OFFSET_____-535x98.html (File name too long)
          Hide
          dsmiley David Smiley added a comment -

          Thanks for bringing this to my attention!

          Ugh, it appears the use of @ParametersFactory means it toString's the constructor args, and Calendar happens to have a long toString. Any suggestions Dawid Weiss?

          Show
          dsmiley David Smiley added a comment - Thanks for bringing this to my attention! Ugh, it appears the use of @ParametersFactory means it toString's the constructor args, and Calendar happens to have a long toString. Any suggestions Dawid Weiss ?
          Hide
          steve_rowe Steve Rowe added a comment -

          See Dawid Weiss's suggestions from the past: http://markmail.org/message/diu2wpjiiyrlfgh6.

          Here's a patch I'm going to try locally:

          diff --git a/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java b/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java
          index d76454e..022c6de 100644
          --- a/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java
          +++ b/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java
          @@ -32,7 +32,7 @@ import org.locationtech.spatial4j.shape.SpatialRelation;
           
           public class DateRangePrefixTreeTest extends LuceneTestCase {
           
          -  @ParametersFactory
          +  @ParametersFactory(argumentFormatting = "calendar=%s")
             public static Iterable<Object[]> parameters() {
               return Arrays.asList(new Object[][]{
                   {DateRangePrefixTree.DEFAULT_CAL}, {DateRangePrefixTree.JAVA_UTIL_TIME_COMPAT_CAL}
          
          Show
          steve_rowe Steve Rowe added a comment - See Dawid Weiss 's suggestions from the past: http://markmail.org/message/diu2wpjiiyrlfgh6 . Here's a patch I'm going to try locally: diff --git a/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java b/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java index d76454e..022c6de 100644 --- a/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java +++ b/lucene/spatial-extras/src/test/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest.java @@ -32,7 +32,7 @@ import org.locationtech.spatial4j.shape.SpatialRelation; public class DateRangePrefixTreeTest extends LuceneTestCase { - @ParametersFactory + @ParametersFactory(argumentFormatting = "calendar=%s") public static Iterable<Object[]> parameters() { return Arrays.asList(new Object[][]{ {DateRangePrefixTree.DEFAULT_CAL}, {DateRangePrefixTree.JAVA_UTIL_TIME_COMPAT_CAL}
          Hide
          dweiss Dawid Weiss added a comment -

          Indeed, use argumentFormatting. You can even use two arguments to the constructor and use only the first one to create the description – then you can format it any way you like inside the parameters factory.

            @ParametersFactory(argumentFormatting = "%s")
             public static Iterable<Object[]> parameters() {
               return Arrays.asList(new Object[][]{
                   {"default", DateRangePrefixTree.DEFAULT_CAL},
                   {"compat", DateRangePrefixTree.JAVA_UTIL_TIME_COMPAT_CAL}
          
          Show
          dweiss Dawid Weiss added a comment - Indeed, use argumentFormatting . You can even use two arguments to the constructor and use only the first one to create the description – then you can format it any way you like inside the parameters factory. @ParametersFactory(argumentFormatting = "%s" ) public static Iterable< Object []> parameters() { return Arrays.asList( new Object [][]{ { " default " , DateRangePrefixTree.DEFAULT_CAL}, { "compat" , DateRangePrefixTree.JAVA_UTIL_TIME_COMPAT_CAL}
          Hide
          steve_rowe Steve Rowe added a comment -

          My patch resulted in the same problem:

          java.io.FileNotFoundException: /home/sarowe/git/lucene-solr/lucene/build/clover/reports/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest_testToStringISO8601__calendar_java_util_GregorianCalendar_time___areFieldsSet_false_areAllFieldsSet_false_lenient_true_zone_sun_util_calendar_ZoneInfo_id__UTC__offset_0_dstSavings_0_useDaylight_false_transitions_0_lastRule_null__firstDayOfWeek_1_minimalDaysInFirstWeek_1_ERA___YEAR___MONTH___WEEK_OF_YEAR___WEEK_OF_MONTH___DAY_OF_MONTH___DAY_OF_YEAR___DAY_OF_WEEK___DAY_OF_WEEK_IN_MONTH___AM_PM___HOUR___HOUR_OF_DAY___MINUTE___SECOND___MILLISECOND___ZONE_OFFSET___DST_OFFSET_____-6fh1l0.html (File name too long)
          

          I'll try Dawid's latest suggestion. Unfortunately it took 273 minutes to run the last time, so it'll be a while...

          Show
          steve_rowe Steve Rowe added a comment - My patch resulted in the same problem: java.io.FileNotFoundException: /home/sarowe/git/lucene-solr/lucene/build/clover/reports/org/apache/lucene/spatial/prefix/tree/DateRangePrefixTreeTest_testToStringISO8601__calendar_java_util_GregorianCalendar_time___areFieldsSet_false_areAllFieldsSet_false_lenient_true_zone_sun_util_calendar_ZoneInfo_id__UTC__offset_0_dstSavings_0_useDaylight_false_transitions_0_lastRule_null__firstDayOfWeek_1_minimalDaysInFirstWeek_1_ERA___YEAR___MONTH___WEEK_OF_YEAR___WEEK_OF_MONTH___DAY_OF_MONTH___DAY_OF_YEAR___DAY_OF_WEEK___DAY_OF_WEEK_IN_MONTH___AM_PM___HOUR___HOUR_OF_DAY___MINUTE___SECOND___MILLISECOND___ZONE_OFFSET___DST_OFFSET_____-6fh1l0.html (File name too long) I'll try Dawid's latest suggestion. Unfortunately it took 273 minutes to run the last time, so it'll be a while...
          Hide
          steve_rowe Steve Rowe added a comment -

          Woohoo! Dawid Weiss's patch worked (with the addition of an unused constructor param) - thanks!:

          BUILD SUCCESSFUL
          Total time: 295 minutes 53 seconds
          

          Here's the patch I'll commit shortly.

          Show
          steve_rowe Steve Rowe added a comment - Woohoo! Dawid Weiss 's patch worked (with the addition of an unused constructor param) - thanks!: BUILD SUCCESSFUL Total time: 295 minutes 53 seconds Here's the patch I'll commit shortly.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 3d22ea3492c0fb81195a9b44b2beec987e1d2aef in lucene-solr's branch refs/heads/master from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3d22ea3 ]

          LUCENE-7278: Get Clover working again

          Show
          jira-bot ASF subversion and git services added a comment - Commit 3d22ea3492c0fb81195a9b44b2beec987e1d2aef in lucene-solr's branch refs/heads/master from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=3d22ea3 ] LUCENE-7278 : Get Clover working again
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 6d530e085701e32e593395ef54ec402ad40a2fb9 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe
          [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6d530e0 ]

          LUCENE-7278: Get Clover working again

          Show
          jira-bot ASF subversion and git services added a comment - Commit 6d530e085701e32e593395ef54ec402ad40a2fb9 in lucene-solr's branch refs/heads/branch_6x from Steve Rowe [ https://git-wip-us.apache.org/repos/asf?p=lucene-solr.git;h=6d530e0 ] LUCENE-7278 : Get Clover working again
          Hide
          dweiss Dawid Weiss added a comment -

          You're very welcome!

          Show
          dweiss Dawid Weiss added a comment - You're very welcome!

            People

            • Assignee:
              dsmiley David Smiley
              Reporter:
              dsmiley David Smiley
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development