Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Why doesn't PIG support partition by on order by?

      If PIG would then PIG could be used for a mess of windowing functions.

      Is there some reason why we can't add a custom partitioner on a order by operation?

      Is there a work around to do windowing in PIG.

      I understand the I can group and then order with in a group, but what if the items with in the group are super big will I have memory issues? Because I need to order the values with in a group. i.e. stock tickers is the group and they need to be sorted on time.

        Activity

        Hide
        Ted Malaska added a comment -

        Here is a simple use case

        I want to add the max and min price for the day up onto the current record

        Ticker|time|price
        FB|1|5
        FB|2|4
        FB|3|6

        would output the following

        Ticker|time|price|max|min
        FB|1|5|5|5
        FB|2|4|5|4
        FB|3|6|6|4

        For every trade of FB in a given day or week.

        Show
        Ted Malaska added a comment - Here is a simple use case I want to add the max and min price for the day up onto the current record Ticker|time|price FB|1|5 FB|2|4 FB|3|6 would output the following Ticker|time|price|max|min FB|1|5|5|5 FB|2|4|5|4 FB|3|6|6|4 For every trade of FB in a given day or week.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ted Malaska
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development