Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-9080

DateMath is broken before the year 1582

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 6.0
    • 6.0.1, 6.1
    • None
    • None

    Description

      In Solr 6.0, dates are parsed using the Java 8 java.time API. It formerly was parsed using java.util.SimpleDateFormat which uses java.util.GregorianCalendar. I've learned that the java.time API does not switch to a different algorithm at the Gregorian Change Date (year 1582) whereas GregorianCalendar does. A ramification of this is that the milliseconds before epoch value is different between the APIs for dates prior to this year. They both round-trip between themselves but not between each other prior to this date. Thus, anyone indexing historical dates must re-index when moving to Solr 6.

      What was not changed in the parsing code was Solr's date-math logic – it still uses the Calendar API. This works for dates after 1582 but before, it'll introduce discrepancies. Here's an example showing weird behavior:
      http://localhost:8983/solr/techproducts/select?facet.range.end=1400-01-01T00:00:00Z&facet.range.gap=%2B10YEARS&facet.range.start=1300-01-01T00:00:00Z/YEAR&facet.range=manufacturedate_dt&facet=on&indent=on&q=*:*&rows=0&wt=json
      Note that the year 1300 rounded down to the year, becomes 1299 January 8th (weird in and of itself) and that subsequent gaps start on the 9th.

      "counts":[
                "1299-01-08T00:00:00Z",0,
                "1309-01-09T00:00:00Z",0,
                "1319-01-09T00:00:00Z",0,  ...
      

      This weirdness will show itself for units at the year or month level, but not below that (from what I'm seeing). In other words, if facet.range.gap is at this amount, or otherwise using the date math syntax to round or add a year or month, there will be issues like this. Otherwise there doesn't seem to be an issue.

      I think the solution is clearly to switch the date math code to use the java.time API.

      Attachments

        Issue Links

          Activity

            People

              dsmiley David Smiley
              dsmiley David Smiley
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: