Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Minor Minor
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      Why doesn't PIG support partition by on order by?

      If PIG would then PIG could be used for a mess of windowing functions.

      Is there some reason why we can't add a custom partitioner on a order by operation?

      Is there a work around to do windowing in PIG.

      I understand the I can group and then order with in a group, but what if the items with in the group are super big will I have memory issues? Because I need to order the values with in a group. i.e. stock tickers is the group and they need to be sorted on time.

        Activity

        Ted Malaska created issue -
        Ted Malaska made changes -
        Field Original Value New Value
        Description Why doesn't PIG support partition by on order by?

        If PIG would then PIG could be used for a mess of windowing functions.

        Is there some reason why we can't add a custom partitioner on a order by operation?

        Is there a work around to do windowing in PIG.

        I understand the I can group and then group with in a group, but what if the items with in the group are super big will I have memory issues? Because I need to order the values with in a group. i.e. stock tickers is the group and they need to be sorted on time.
        Why doesn't PIG support partition by on order by?

        If PIG would then PIG could be used for a mess of windowing functions.

        Is there some reason why we can't add a custom partitioner on a order by operation?

        Is there a work around to do windowing in PIG.

        I understand the I can group and then order with in a group, but what if the items with in the group are super big will I have memory issues? Because I need to order the values with in a group. i.e. stock tickers is the group and they need to be sorted on time.
        Hide
        Ted Malaska added a comment -

        Here is a simple use case

        I want to add the max and min price for the day up onto the current record

        Ticker|time|price
        FB|1|5
        FB|2|4
        FB|3|6

        would output the following

        Ticker|time|price|max|min
        FB|1|5|5|5
        FB|2|4|5|4
        FB|3|6|6|4

        For every trade of FB in a given day or week.

        Show
        Ted Malaska added a comment - Here is a simple use case I want to add the max and min price for the day up onto the current record Ticker|time|price FB|1|5 FB|2|4 FB|3|6 would output the following Ticker|time|price|max|min FB|1|5|5|5 FB|2|4|5|4 FB|3|6|6|4 For every trade of FB in a given day or week.

          People

          • Assignee:
            Unassigned
            Reporter:
            Ted Malaska
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:

              Development