Pig
  1. Pig
  2. PIG-1793

Add macro expansion to Pig Latin

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.9.0
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      As production Pig scripts grow longer and longer, Pig Latin has a need to integrate standard programming techniques of separation and code sharing offered by functions and modules. A proposal of adding macro expansion to Pig Latin is posted here: http://wiki.apache.org/pig/TuringCompletePig

      Below is a brief summary of the proposed syntax (and examples):

      • Macro Definition

      The existing DEFINE keyword will be expanded to allow definitions of Pig macros.

      Syntax

      define <name> (<params>) returns <aliases> {
          <Pig Latin fragment>
      };
      

      Example

      define my_macro(A, sortkey) returns C {
          B = filter $A by my_filter(*);
          $C = order B by $sortkey;
      }
      
      • Macro Expansion

      Syntax

      <aliases> = <macro name> (<params>);
      

      Example: Use above macro in a Pig script:

      X = load 'foo' as (user, address, phone);
      Y = my_macro(X, user);
      store Y into 'bar';
      

      This script is expanded into the following Pig Latin statements:

      X = load 'foo' as (user, address, phone);
      macro_my_macro_B_1 = filter X by my_filter(*);
      Y = order macro_my_macro_B_1 by user;
      store Y into 'bar';
      

      Notes

      1. Any alias in the macro which isn't visible from outside will be prefixed with macro name and suffixed with instance id to avoid namespace collision.
      2. Macro expansion is not a complete replacement for function calls. Recursive expansions are not supported.

      • Macro Import

      The new IMPORT keyword can be used to add macros defined in another Pig Latin file.

      Syntax

      import <Pig Latin file name>;
      

      Example

      import my_macro.pig;
      

      Note: All macro names are in the global namespace.

      1. PIG-1793_1.patch
        88 kB
        Richard Ding
      2. PIG-1793_2.patch
        19 kB
        Richard Ding
      3. PIG-1793.patch
        85 kB
        Richard Ding

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Richard Ding
              Reporter:
              Richard Ding
            • Votes:
              1 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development