Pig
  1. Pig
  2. PIG-3317

disable optimizations via pig properties

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.12.0
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None
    • Release Note:
      Hide
      Optimization rules can be disabled with the "pig.optimizer.rules.disabled" pig property, allowing scripts to disable optimizations themselves. As in prior release, optimizations may still be disabled with the "optimizer_off" command-line option.
      Show
      Optimization rules can be disabled with the "pig.optimizer.rules.disabled" pig property, allowing scripts to disable optimizations themselves. As in prior release, optimizations may still be disabled with the "optimizer_off" command-line option.

      Description

      Pig provides a number of optimizations which are described at http://pig.apache.org/docs/r0.11.1/perf.html#optimization-rules. As is described in the docs, all or specific optimizations can be disabled via the command-line.

      Currently the caller of a pig script must know which optimizations to disable when running because that information cannot be set in the script itself. Nor can optimizations be disabled site-wide through pig.properties.

      Pig should allow disabling optimizations via properties so that pig scripts themselves can disable optimizations as needed, rather than the caller needing to know what optimizations to disable on the command-line.

      1. PIG-3317_disable_opts.1.patch
        9 kB
        Travis Crawford
      2. PIG-3317_disable_opts.2.patch
        14 kB
        Travis Crawford
      3. PIG-3317_disable_opts.3.patch
        17 kB
        Travis Crawford
      4. PIG-3317_disable_opts.4.patch
        17 kB
        Travis Crawford

        Activity

        Hide
        Travis Crawford added a comment -
        Show
        Travis Crawford added a comment - Review board: https://reviews.apache.org/r/11032/
        Hide
        Bill Graham added a comment -

        Commented as such in the rb, but this patch looks good to me.

        Show
        Bill Graham added a comment - Commented as such in the rb, but this patch looks good to me.
        Hide
        Bill Graham added a comment -

        Canceling patch since Travis and Julien identified issues with SET in scripts in https://reviews.apache.org/r/11032/.

        Show
        Bill Graham added a comment - Canceling patch since Travis and Julien identified issues with SET in scripts in https://reviews.apache.org/r/11032/ .
        Hide
        Travis Crawford added a comment -

        Updated patch; please see review for comments. Main change is enabling scripts to:

        set pig.optimizer.rules.disabled 'ColumnMapKeyPrune';
        Show
        Travis Crawford added a comment - Updated patch; please see review for comments. Main change is enabling scripts to: set pig.optimizer.rules.disabled 'ColumnMapKeyPrune';
        Hide
        Travis Crawford added a comment -

        Changes per review.

        Show
        Travis Crawford added a comment - Changes per review.
        Hide
        Bill Graham added a comment -

        Committed, thanks Travis!

        Show
        Bill Graham added a comment - Committed, thanks Travis!

          People

          • Assignee:
            Travis Crawford
            Reporter:
            Travis Crawford
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development