Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: core/queryparser
    • Labels:
      None
    • Lucene Fields:
      Patch Available

      Description

      The QueryParser currently uses the deprecated class DateField to create RangeQueries with date values. However, most users probably use DateTools to store date values in their indexes, because this is the recommended way since DateField has been deprecated. In that case RangeQueries with date values produced by the QueryParser won't work with those indexes.

      This patch replaces the use of DateField in QueryParser by DateTools. Because DateTools can produce date values with different resolutions, this patch adds the following methods to QueryParser:

      /**

      • Sets the default date resolution used by RangeQueries for fields for which no
      • specific date resolutions has been set. Field specific resolutions can be set
      • with {@link #setDateResolution(String, DateTools.Resolution)}

        .

      • @param dateResolution the default date resolution to set
        */
        public void setDateResolution(DateTools.Resolution dateResolution);

      /**

      • Sets the date resolution used by RangeQueries for a specific field.
      • @param field field for which the date resolution is to be set
      • @param dateResolution date resolution to set
        */
        public void setDateResolution(String fieldName, DateTools.Resolution dateResolution);

      (I also added the corresponding getter methods).

      Now the user can set a default date resolution used for all fields or, with the second method, field specific date resolutions.
      The initial default resolution, which is used if the user does not set a different resolution, is DateTools.Resolution.DAY.

      Please let me know if you think we should use a different resolution as default.

      I extended TestQueryParser to test this new feature.

      All unit tests pass.

      1. queryparser_datetools2.patch
        17 kB
        Michael Busch
      2. queryparser_datetools.patch
        16 kB
        Michael Busch

        Activity

        Hide
        Hoss Man added a comment -

        I've commited queryparser_datetools2.patch with two changes...

        1) revised the class level javadocs based on Michael's last comment.
        2) reinstated the old TestQueryParser.testDateRange() as testLegacyDateRange() to attempt to future proof against breaking backwards compatibility.

        Show
        Hoss Man added a comment - I've commited queryparser_datetools2.patch with two changes... 1) revised the class level javadocs based on Michael's last comment. 2) reinstated the old TestQueryParser.testDateRange() as testLegacyDateRange() to attempt to future proof against breaking backwards compatibility.
        Hide
        Michael Busch added a comment -

        You're right Hoss, the word "format" is used ambiguously in the javadoc. We could change it to

        • <p>In {@link RangeQuery}

          s, QueryParser tries to detect date values, e.g. <tt>date:[6/1/2005 TO 6/4/2005]</tt>

        • produces a range query that searches for "date" fields between 2005-06-01 and 2005-06-04. Note
        • that the format of the accepted input depends on {@link #setLocale(Locale) the locale}

          .

        • By default a date is converted into a search term using the deprecated {@link DateField} for compatibility reasons.
          * To use the new {@link DateTools} to convert dates, a {@link DateTools.Resolution} has to be set.</p>
          * <p>The date resolution that shall be used for RangeQueries can be set using {@link #setDateResolution(DateTools.Resolution)}
          * or {@link #setDateResolution(String, DateTools.Resolution)}. The former sets the default date resolution for all fields, whereas the latter can
          * be used to set field specific date resolutions. Field specific date resolutions take, if set, precedence over the default date resolution.
          * </p>
          * <p>If you use neither {@link DateField}

          nor

          {@link DateTools}

          in your index, you can create your own

        • query parser that inherits QueryParser and overwrites {@link #getRangeQuery(String, String, String, boolean)}

          to

        • use a different method for date conversion.
        • </p>

        Sounds better? Do you want me to create another patch that includes this javadoc?

        Show
        Michael Busch added a comment - You're right Hoss, the word "format" is used ambiguously in the javadoc. We could change it to <p>In {@link RangeQuery} s, QueryParser tries to detect date values, e.g. <tt>date: [6/1/2005 TO 6/4/2005] </tt> produces a range query that searches for "date" fields between 2005-06-01 and 2005-06-04. Note that the format of the accepted input depends on {@link #setLocale(Locale) the locale} . By default a date is converted into a search term using the deprecated {@link DateField} for compatibility reasons. * To use the new {@link DateTools} to convert dates, a {@link DateTools.Resolution} has to be set.</p> * <p>The date resolution that shall be used for RangeQueries can be set using {@link #setDateResolution(DateTools.Resolution)} * or {@link #setDateResolution(String, DateTools.Resolution)}. The former sets the default date resolution for all fields, whereas the latter can * be used to set field specific date resolutions. Field specific date resolutions take, if set, precedence over the default date resolution. * </p> * <p>If you use neither {@link DateField} nor {@link DateTools} in your index, you can create your own query parser that inherits QueryParser and overwrites {@link #getRangeQuery(String, String, String, boolean)} to use a different method for date conversion. </p> Sounds better? Do you want me to create another patch that includes this javadoc?
        Hide
        Hoss Man added a comment -

        +1 for queryparser_datetools2.patch

        the only nitpick i have is with the class level javadocs...

        • <p>In {@link RangeQuery}

          s, QueryParser tries to detect date values, e.g. <tt>date:[6/1/2005 TO 6/4/2005]</tt>

        • produces a range query that searches for "date" fields between 2005-06-01 and 2005-06-04. Note
        • that the format of the accepted input depends on {@link #setLocale(Locale) the locale}

          .

        • By default a date is formatted using the deprecated {@link DateField}

          for compatibility reasons.

        • To use the new {@link DateTools}

          to format dates, a

          {@link DateTools.Resolution}

          has to be set.</p>

        • <p>The date resolution that shall be used for RangeQueries can be set using {@link #setDateResolution(DateTools.Resolution)

        ...the word "format" is used to mean two very differnet things in this paragraph, we should clean that up so it's clera that Locale determines the DateFormat QP uses when trying to parse input in [a TO b] syntax, while either DateField or DateTools are used to create the "indexed" value queried against.

        Show
        Hoss Man added a comment - +1 for queryparser_datetools2.patch the only nitpick i have is with the class level javadocs... <p>In {@link RangeQuery} s, QueryParser tries to detect date values, e.g. <tt>date: [6/1/2005 TO 6/4/2005] </tt> produces a range query that searches for "date" fields between 2005-06-01 and 2005-06-04. Note that the format of the accepted input depends on {@link #setLocale(Locale) the locale} . By default a date is formatted using the deprecated {@link DateField} for compatibility reasons. To use the new {@link DateTools} to format dates, a {@link DateTools.Resolution} has to be set.</p> <p>The date resolution that shall be used for RangeQueries can be set using {@link #setDateResolution(DateTools.Resolution) ...the word "format" is used to mean two very differnet things in this paragraph, we should clean that up so it's clera that Locale determines the DateFormat QP uses when trying to parse input in [a TO b] syntax, while either DateField or DateTools are used to create the "indexed" value queried against.
        Hide
        Michael Busch added a comment -

        Changed the summary to better reflect the second version of the patch.

        Show
        Michael Busch added a comment - Changed the summary to better reflect the second version of the patch.
        Hide
        Michael Busch added a comment -

        This new patch contains the changes suggested by Hoss:

        • by default the QueryParser uses DateField to format dates for all fields
        • if a date resolution has been set using void setDateResolution(DateTools.Resolution) then DateTools is used for all fields with the given resolution
        • if a field specific date resolution has been set using setDateResolution(String, DateTools.Resolution) then DateTools is used for the given field with the given date resolution; other fields are not affected

        So with this patch the behaviour of QueryParser does not change unless either setDateResolution(DateTools.Resolution) or setDateResolution(String, DateTools.Resolution) have been called.

        Show
        Michael Busch added a comment - This new patch contains the changes suggested by Hoss: by default the QueryParser uses DateField to format dates for all fields if a date resolution has been set using void setDateResolution(DateTools.Resolution) then DateTools is used for all fields with the given resolution if a field specific date resolution has been set using setDateResolution(String, DateTools.Resolution) then DateTools is used for the given field with the given date resolution; other fields are not affected So with this patch the behaviour of QueryParser does not change unless either setDateResolution(DateTools.Resolution) or setDateResolution(String, DateTools.Resolution) have been called.
        Hide
        Michael Busch added a comment -

        Actually it is documented in the QueryParser how to use a different format for dates:

        ...

        • feature also assumes that your index uses the {@link DateTools}

          class to store dates.

        • If you use a different format and you still want QueryParser
        • to turn local dates in range queries into valid queries you need to create your own
        • query parser that inherits QueryParser and overwrites
        • {@link #getRangeQuery(String, String, String, boolean)}

          .
          ...

        And the javadoc of DateField says:

        Deprecated. If you build a new index, use DateTools instead. This class is included for use with existing indices and will be removed in a future release.

        So the question is in how many future releases we want to support DateField. If we still want to support it in 2.1 I agree to Hoss that his suggestion would be a nice and clean way to do that and I can easily change the patch accordingly. It avoids having another option in QueryParser.

        Show
        Michael Busch added a comment - Actually it is documented in the QueryParser how to use a different format for dates: ... feature also assumes that your index uses the {@link DateTools} class to store dates. If you use a different format and you still want QueryParser to turn local dates in range queries into valid queries you need to create your own query parser that inherits QueryParser and overwrites {@link #getRangeQuery(String, String, String, boolean)} . ... And the javadoc of DateField says: Deprecated. If you build a new index, use DateTools instead. This class is included for use with existing indices and will be removed in a future release. So the question is in how many future releases we want to support DateField. If we still want to support it in 2.1 I agree to Hoss that his suggestion would be a nice and clean way to do that and I can easily change the patch accordingly. It avoids having another option in QueryParser.
        Hide
        Hoss Man added a comment -

        cleanest way to be backwards compatible would be to not have an initial default Resolution, and use DateField if no Resolution can be found for the field specified. Existing clients would still get DateField for all dates, clients that add a call to setDateResolution("foo", BAR); would get a DateTools formated query with resolution BAR for field foo, but other pre-existing fields would still use DateField, and clients that call setDateResolution(BAR); would allways get a DateTools formatted query, either at resolution BAR or at some other resolution if they also use the two arg setDateResolution

        Show
        Hoss Man added a comment - cleanest way to be backwards compatible would be to not have an initial default Resolution, and use DateField if no Resolution can be found for the field specified. Existing clients would still get DateField for all dates, clients that add a call to setDateResolution("foo", BAR); would get a DateTools formated query with resolution BAR for field foo, but other pre-existing fields would still use DateField, and clients that call setDateResolution(BAR); would allways get a DateTools formatted query, either at resolution BAR or at some other resolution if they also use the two arg setDateResolution
        Hide
        Daniel Naber added a comment -

        I'm not sure if most people use DateTools already, as it has just been added in Lucene 1.9. Maybe you could consider an option (yes, yet another option isn't nice, I know)? Otherwise we need to properly document how to continue using DateField, i.e. by extending QueryParser and overwriting this method I guess.

        Show
        Daniel Naber added a comment - I'm not sure if most people use DateTools already, as it has just been added in Lucene 1.9. Maybe you could consider an option (yes, yet another option isn't nice, I know)? Otherwise we need to properly document how to continue using DateField, i.e. by extending QueryParser and overwriting this method I guess.

          People

          • Assignee:
            Michael Busch
            Reporter:
            Michael Busch
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development