Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.92.0
    • Component/s: Coprocessors
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      A coprocessor to do basic aggregating; max, min, counts, etc.

      Description

      Chatting with jgray and holstad at the kitchen table about counts, sums, and other aggregating facility, facility generally where you want to calculate some meta info on your table, it seems like it wouldn't be too hard making a filter type that could run a function server-side and return the result ONLY of the aggregation or whatever.

      For example, say you just want to count rows, currently you scan, server returns all data to client and count is done by client counting up row keys. A bunch of time and resources have been wasted returning data that we're not interested in. With this new filter type, the counting would be done server-side and then it would make up a new result that was the count only (kinda like mysql when you ask it to count, it returns a 'table' with a count column whose value is count of rows). We could have it so the count was just done per region and return that. Or we could maybe make a small change in scanner too so that it aggregated the per-region counts.

      1. patch-1512-9.txt
        61 kB
        Himanshu Vashishtha
      2. patch-1512-8.txt
        71 kB
        Himanshu Vashishtha
      3. patch-1512-7.txt
        61 kB
        Himanshu Vashishtha
      4. patch-1512-6.txt
        61 kB
        Himanshu Vashishtha
      5. patch-1512-5.txt
        61 kB
        Himanshu Vashishtha
      6. patch-1512-4.txt
        61 kB
        Himanshu Vashishtha
      7. patch-1512-3.txt
        60 kB
        Himanshu Vashishtha
      8. patch-1512-2.txt
        46 kB
        Himanshu Vashishtha
      9. patch-1512.txt
        25 kB
        Himanshu Vashishtha
      10. ColumnInterpreter.java
        2 kB
        Ted Yu
      11. AggregationClient.java
        13 kB
        Ted Yu
      12. AggregateProtocolImpl.java
        10 kB
        Ted Yu
      13. AggregateCpProtocol.java
        4 kB
        Ted Yu
      14. addendum_1512.txt
        2 kB
        Himanshu Vashishtha
      15. 1512.zip
        17 kB
        Himanshu Vashishtha

        Issue Links

          Activity

          Jeff Hammerbacher made changes -
          Link This issue is related to HBASE-4435 [ HBASE-4435 ]
          Himanshu Vashishtha made changes -
          Attachment addendum_1512.txt [ 12477446 ]
          stack made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Hadoop Flags [Reviewed]
          Release Note A coprocessor to do basic aggregating; max, min, counts, etc.
          Fix Version/s 0.92.0 [ 12314223 ]
          Resolution Fixed [ 1 ]
          stack made changes -
          Assignee Himanshu Vashishtha [ v.himanshu ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-9.txt [ 12477332 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-8.txt [ 12477327 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-7.txt [ 12477212 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-6.txt [ 12476629 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-5.txt [ 12476216 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-4.txt [ 12475627 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-3.txt [ 12475271 ]
          Ted Yu made changes -
          Attachment AggregateCpProtocol.java [ 12475221 ]
          Attachment AggregateProtocolImpl.java [ 12475222 ]
          Attachment AggregationClient.java [ 12475223 ]
          Attachment ColumnInterpreter.java [ 12475224 ]
          Ted Yu made changes -
          Attachment ColumnInterpreter.java [ 12475133 ]
          Ted Yu made changes -
          Attachment AggregationClient.java [ 12475134 ]
          Ted Yu made changes -
          Attachment AggregateProtocolImpl.java [ 12475131 ]
          Ted Yu made changes -
          Attachment AggregateCpProtocol.java [ 12475135 ]
          Ted Yu made changes -
          Attachment AggregateCpProtocol.java [ 12475135 ]
          Ted Yu made changes -
          Attachment AggregationClient.java [ 12475130 ]
          Ted Yu made changes -
          Attachment ColumnInterpreter.java [ 12475133 ]
          Attachment AggregationClient.java [ 12475134 ]
          Ted Yu made changes -
          Attachment ColumnInterpreter.java [ 12475129 ]
          Ted Yu made changes -
          Attachment AggregateCpProtocol.java [ 12475132 ]
          Ted Yu made changes -
          Attachment ColumnInterpreter.java [ 12475129 ]
          Attachment AggregationClient.java [ 12475130 ]
          Attachment AggregateProtocolImpl.java [ 12475131 ]
          Attachment AggregateCpProtocol.java [ 12475132 ]
          Ted Yu made changes -
          Attachment AggregationClient.java [ 12475056 ]
          Ted Yu made changes -
          Attachment AggregationClient.java [ 12475056 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512-2.txt [ 12474727 ]
          Todd Lipcon made changes -
          Component/s coprocessors [ 12314191 ]
          Himanshu Vashishtha made changes -
          Attachment patch-1512.txt [ 12464830 ]
          Himanshu Vashishtha made changes -
          Attachment 1512.zip [ 12458533 ]
          Andrew Purtell made changes -
          Summary Coprocessors: Support filters that implement aggregate functions Coprocessors: Support aggregate functions
          Andrew Purtell made changes -
          Summary Coprocessors: Filter/aggregate functions Coprocessors: Support filters that implement aggregate functions
          Andrew Purtell made changes -
          Summary New filter type: a filter that can return results that it has fabricated (use for doing sum, count, etc.) Coprocessors: Filter/aggregate functions
          Andrew Purtell made changes -
          Field Original Value New Value
          Parent HBASE-2000 [ 12441373 ]
          Issue Type Improvement [ 4 ] Sub-task [ 7 ]
          stack created issue -

            People

            • Assignee:
              Himanshu Vashishtha
              Reporter:
              stack
            • Votes:
              4 Vote for this issue
              Watchers:
              19 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development