Uploaded image for project: 'Solr'
  1. Solr
  2. SOLR-7707

Add StreamExpression Support to RollupStream

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Closed
    • Minor
    • Resolution: Fixed
    • None
    • 6.0
    • SolrJ
    • None

    Description

      This ticket is to add Stream Expression support to the RollupStream as discussed in SOLR-7560.

      Proposed expression syntax for the RollupStream (copied from that ticket)

      rollup(
        someStream(....),
        over="fieldA, fieldB, fieldC",
        min(fieldA),
        max(fieldA),
        min(fieldB),
        mean(fieldD),
        sum(fieldC)
      )
      

      This requires making the *Metric types Expressible but I think that ends up as a good thing. Would make it real easy to support other options on metrics like excluding outliers, for example find the sum of values within 3 standard deviations from the mean could be

      sum(fieldC, limit=standardDev(3))
      

      (note, how that particular calculation could be implemented is left as an exercise for the reader, I'm just using it as an example of adding additional options on a relatively simple metric).
      Another option example is what to do with null values. For example, in some cases a null should not impact a mean but in others it should. You could express those as

      mean(fieldA, replace(null, 0))  // replace null values with 0 thus leading to an impact on the mean
      mean(fieldA, includeNull="true") // nulls are counted in the denominator but nothing added to numerator
      mean(fieldA, includeNull="false") // nulls neither counted in denominator nor added to numerator
      mean(fieldA, replace(null, fieldB), includeNull="true") // if fieldA is null replace it with fieldB, include null fieldB in mean
      

      so on and so forth.

      Attachments

        1. SOLR-7707.patch
          65 kB
          Dennis Gove
        2. SOLR-7707.patch
          75 kB
          Dennis Gove
        3. SOLR-7707.patch
          75 kB
          Dennis Gove
        4. SOLR-7707.patch
          74 kB
          Joel Bernstein

        Activity

          People

            Unassigned Unassigned
            dpgove Dennis Gove
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: