Pig
  1. Pig
  2. PIG-3010

Allow UDF's to flatten themselves

    Details

    • Type: Improvement Improvement
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: None
    • Labels:
      None

      Description

      This is something I thought would be cool for a while, so I sat down and did it because I think there are some useful debugging tools it'd help with.

      The idea is that if you attach an annotation to a UDF, the Tuple or DataBag you output will be flattened. This is quite powerful. A very common pattern is:

      a = foreach data generate Flatten(MyUdf(thing)) as (a,b,c);

      This would let you just do:

      a = foreach data generate MyUdf(thing);

      With the exact same result!

      1. PIG-3010-0.patch
        79 kB
        Jonathan Coveney
      2. PIG-3010-1.patch
        71 kB
        Jonathan Coveney
      3. PIG-3010-2.patch
        285 kB
        Jonathan Coveney
      4. PIG-3010-2_nowhitespace.patch
        68 kB
        Jonathan Coveney
      5. PIG-3010-3_nows.patch
        69 kB
        Jonathan Coveney
      6. PIG-3010-3.patch
        261 kB
        Jonathan Coveney
      7. PIG-3010-4_nows.patch
        69 kB
        Jonathan Coveney
      8. PIG-3010-4.patch
        261 kB
        Jonathan Coveney
      9. PIG-3010-5_nows.patch
        69 kB
        Jonathan Coveney
      10. PIG-3010-5.patch
        261 kB
        Jonathan Coveney

        Issue Links

          Activity

          Rohini Palaniswamy made changes -
          Fix Version/s 0.17.0 [ 12335654 ]
          Fix Version/s 0.16.0 [ 12332168 ]
          Daniel Dai made changes -
          Fix Version/s 0.16.0 [ 12332168 ]
          Fix Version/s 0.15.0 [ 12328760 ]
          Daniel Dai made changes -
          Fix Version/s 0.15.0 [ 12328760 ]
          Fix Version/s 0.14.0 [ 12326954 ]
          Aniket Mokashi made changes -
          Fix Version/s 0.14.0 [ 12326954 ]
          Fix Version/s 0.13.0 [ 12324971 ]
          Daniel Dai made changes -
          Fix Version/s 0.13.0 [ 12324971 ]
          Fix Version/s 0.12.0 [ 12323380 ]
          Gavin made changes -
          Link This issue is depended upon by PIG-3088 [ PIG-3088 ]
          Gavin made changes -
          Link This issue blocks PIG-3088 [ PIG-3088 ]
          Alan Gates made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Jonathan Coveney made changes -
          Attachment PIG-3010-5_nows.patch [ 12570284 ]
          Attachment PIG-3010-5.patch [ 12570285 ]
          Jonathan Coveney made changes -
          Attachment PIG-3010-4_nows.patch [ 12569836 ]
          Attachment PIG-3010-4.patch [ 12569837 ]
          Jonathan Coveney made changes -
          Attachment PIG-3010-3_nows.patch [ 12566053 ]
          Attachment PIG-3010-3.patch [ 12566054 ]
          Jonathan Coveney made changes -
          Attachment PIG-3010-2_nowhitespace.patch [ 12560703 ]
          Jonathan Coveney made changes -
          Attachment PIG-3010-2.patch [ 12560306 ]
          Jonathan Coveney made changes -
          Link This issue blocks PIG-3088 [ PIG-3088 ]
          Jonathan Coveney made changes -
          Attachment PIG-3010-1.patch [ 12550997 ]
          Jonathan Coveney made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Jonathan Coveney made changes -
          Field Original Value New Value
          Attachment PIG-3010-0.patch [ 12550995 ]
          Jonathan Coveney created issue -

            People

            • Assignee:
              Jonathan Coveney
              Reporter:
              Jonathan Coveney
            • Votes:
              2 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:

                Development