Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-4128

New logical optimizer rule: ConstantCalculator

    XMLWordPrintableJSON

Details

    • New Feature
    • Status: Closed
    • Major
    • Resolution: Fixed
    • None
    • 0.14.0
    • impl
    • None
    • Reviewed

    Description

      Pig used to have a LogicExpressionSimplifier to simplify expression which also calculates constant expression. The optimizer rule is buggy and we disable it by default in PIG-2316.

      However, we do need this feature especially in partition/predicate push down, since both does not deal with complex constant expression, we'd like to replace the expression with constant before the actual push down. Yes, user may manually do the calculation and rewrite the query, but even rewrite is sometimes not possible. Consider the case user want to push a datetime predicate, user have to write a ToDate udf since Pig does not have datetime constant.

      In this Jira, I provide a new rule: ConstantCalculator, which is much simpler and much less error prone, to replace LogicExpressionSimplifier.

      Attachments

        1. PIG-4128-3.patch
          219 kB
          Daniel Dai
        2. PIG-4128-2.patch
          220 kB
          Daniel Dai
        3. PIG-4128-1.patch
          219 kB
          Daniel Dai

        Issue Links

          Activity

            People

              daijy Daniel Dai
              daijy Daniel Dai
              Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: