Pig
  1. Pig
  2. PIG-2547

Easier UDFs: Convenient EvalFunc super-classes

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11
    • Component/s: None
    • Labels:
      None
    • Release Note:
      New superclasses to make it easier to implement UDFs.

      Description

      We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.

      There are 3 classes, each extending the next. Class naming is all TBD.

      • TypedOutputEvalFunc<OUT> - Implements public Schema outputSchema(Schema input) based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
      • PrimitiveEvalFunc<IN, OUT> - Same as above with helper validation allowing the ability it subclass and just implement public OUT exec(IN input), where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
      • FunctionWrapperEvalFunc - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where MyFunction is a class that implements Function:
      DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
      
      1. PIG-2547.3.patch
        33 kB
        Bill Graham
      2. PIG_2547.2.patch
        33 kB
        Bill Graham
      3. PIG_2547.1.patch
        38 kB
        Bill Graham

        Activity

        Bill Graham created issue -
        Bill Graham made changes -
        Field Original Value New Value
        Description We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.

        There are classes, each extending the next. Class naming is all TBD.

        * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
        * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
        * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:

        {noformat}
        DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
        {noformat}
         

        We've got a few abstract extensions of EvalFunc that make life easier. If people are interested we can push said classes into Pig.

        There are 3 classes, each extending the next. Class naming is all TBD.

        * {{TypedOutputEvalFunc<OUT>}} - Implements {{public Schema outputSchema(Schema input)}} based on the generic type of the subclass. Provides common helper validation functions which increment counters for good and bad Tuple data passed. Useful where the input to be worked on is a tuple of size N or greater.
        * {{PrimitiveEvalFunc<IN, OUT>}} - Same as above with helper validation allowing the ability it subclass and just implement {{public OUT exec(IN input)}}, where IN and OUT are primitives. Useful when the input is a single primitive in position 0 of a tuple.
        * {{FunctionWrapperEvalFunc}} - Wraps a Guava Function implementation (http://guava-libraries.googlecode.com/svn/trunk/javadoc/com/google/common/base/Function.html) and allows UDFs to be used in Pig scripts like so, where {{MyFunction}} is a class that implements {{Function}}:

        {noformat}
        DEFINE myUdf org.apache.pig.FunctionWrapperEvalFunc('MyFunction')
        {noformat}
         

        Bill Graham made changes -
        Attachment PIG_2547.1.patch [ 12515676 ]
        Bill Graham made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Bill Graham made changes -
        Attachment PIG_2547.2.patch [ 12516197 ]
        Jonathan Coveney made changes -
        Fix Version/s 0.10 [ 12316246 ]
        Fix Version/s 0.11 [ 12318878 ]
        Jonathan Coveney made changes -
        Labels 0.10_blocker
        Daniel Dai made changes -
        Fix Version/s 0.10.1 [ 12320547 ]
        Fix Version/s 0.10.0 [ 12316246 ]
        Bill Graham made changes -
        Labels 0.10_blocker
        Bill Graham made changes -
        Attachment PIG-2547.3.patch [ 12526579 ]
        Bill Graham made changes -
        Fix Version/s 0.10.1 [ 12320547 ]
        Bill Graham made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Release Note New superclasses to make it easier to implement UDFs.
        Resolution Fixed [ 1 ]
        Bill Graham made changes -
        Status Resolved [ 5 ] Closed [ 6 ]

          People

          • Assignee:
            Bill Graham
            Reporter:
            Bill Graham
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development