Solr
  1. Solr
  2. SOLR-1709

Distributed Date and Range Faceting

    Details

      Description

      This patch is for adding support for date facets when using distributed searches.

      Date faceting across multiple machines exposes some time-based issues that anyone interested in this behaviour should be aware of:
      Any time and/or time-zone differences are not accounted for in the patch (i.e. merged date facets are at a time-of-day, not necessarily at a universal 'instant-in-time', unless all shards are time-synced to the exact same time).
      The implementation uses the first encountered shard's facet_dates as the basis for subsequent shards' data to be merged in.
      This means that if subsequent shards' facet_dates are skewed in relation to the first by >1 'gap', these 'earlier' or 'later' facets will not be merged in.
      There are several reasons for this:

      • Performance: It's faster to check facet_date lists against a single map's data, rather than against each other, particularly if there are many shards
      • If 'earlier' and/or 'later' facet_dates are added in, this will make the time range larger than that which was requested
        (e.g. a request for one hour's worth of facets could bring back 2, 3 or more hours of data)
        This could be dealt with if timezone and skew information was added, and the dates were normalized.
        One possibility for adding such support is to [optionally] add 'timezone' and 'now' parameters to the 'facet_dates' map. This would tell requesters what time and TZ the remote server thinks it is, and so multiple shards' time data can be normalized.

      The patch affects 2 files in the Solr core:
      org.apache.solr.handler.component.FacetComponent.java
      org.apache.solr.handler.component.ResponseBuilder.java

      The main changes are in FacetComponent - ResponseBuilder is just to hold the completed SimpleOrderedMap until the finishStage.
      One possible enhancement is to perhaps make this an optional parameter, but really, if facet.date parameters are specified, it is assumed they are desired.
      Comments & suggestions welcome.

      As a favour to ask, if anyone could take my 2 source files and create a PATCH file from it, it would be greatly appreciated, as I'm having a bit of trouble with svn (don't shoot me, but my environment is a Redmond-based os company).

      1. SOLR-1709_3x.patch
        15 kB
        Simon Willnauer
      2. SOLR-1709.patch
        11 kB
        Hoss Man
      3. SOLR-1709_distributed_date_faceting_v3x.patch
        8 kB
        David Smiley
      4. solr-1.4.0-solr-1709.patch
        5 kB
        Thomas Hammerl
      5. FacetComponent.java
        27 kB
        Peter Sturge
      6. ResponseBuilder.java
        8 kB
        Peter Sturge
      7. FacetComponent.java
        27 kB
        Peter Sturge

        Issue Links

          Activity

          Peter Sturge created issue -
          Peter Sturge made changes -
          Field Original Value New Value
          Attachment FacetComponent.java [ 12429740 ]
          Attachment ResponseBuilder.java [ 12429741 ]
          Peter Sturge made changes -
          Attachment FacetComponent.java [ 12431048 ]
          Thomas Hammerl made changes -
          Attachment solr-1.4.0-solr-1709.patch [ 12436092 ]
          Peter Sturge made changes -
          Link This issue is related to SOLR-1729 [ SOLR-1729 ]
          David Smiley made changes -
          Hoss Man made changes -
          Attachment SOLR-1709.patch [ 12476407 ]
          Hoss Man made changes -
          Assignee Hoss Man [ hossman ]
          Fix Version/s 3.2 [ 12316172 ]
          Fix Version/s 4.0 [ 12314992 ]
          Hoss Man made changes -
          Fix Version/s 3.2 [ 12316172 ]
          Hoss Man made changes -
          Summary Distributed Date Faceting Distributed Date and Range Faceting
          Hoss Man made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Simon Willnauer made changes -
          Attachment SOLR-1709_3x.patch [ 12510821 ]
          Simon Willnauer made changes -
          Resolution Fixed [ 1 ]
          Status Resolved [ 5 ] Reopened [ 4 ]
          Assignee Hoss Man [ hossman ] Simon Willnauer [ simonw ]
          Simon Willnauer made changes -
          Fix Version/s 3.6 [ 12319065 ]
          Simon Willnauer made changes -
          Status Reopened [ 4 ] Resolved [ 5 ]
          Resolution Fixed [ 1 ]
          Uwe Schindler made changes -
          Status Resolved [ 5 ] Closed [ 6 ]

            People

            • Assignee:
              Simon Willnauer
              Reporter:
              Peter Sturge
            • Votes:
              2 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development