Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-3545

Seperate validation rules from optimizer

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.13.0
    • Component/s: impl
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      A continuation of PIG-3492. We realize separate validation rules (such as DuplicateForEachColumnRewriteVisitor, ImplicitSplitInsertVisitor) from optimizer would make code clear. The reason is:
      1. validation rule only need to run once
      2. In contrast to optimization rule, validation rule works on an inconsistent plan, and should be careful about precondition. Also some global operation such as resetSchema is not possible on inconsistent plan
      3. Some operations should work with unoptimized but validated plan, such as describe

      Also want to address PIG-3508 in the cleanup. We should call LogicalPlan.optimize explicitly rather than rely on HExecutionEngine.compile to do the logical plan optimization.

        Attachments

        1. PIG-3545-1.patch
          49 kB
          Jianyong Dai
        2. PIG-3545-2.patch
          56 kB
          Jianyong Dai
        3. PIG-3545-3.patch
          57 kB
          Jianyong Dai

          Issue Links

            Activity

              People

              • Assignee:
                daijy Jianyong Dai
                Reporter:
                daijy Jianyong Dai
              • Votes:
                0 Vote for this issue
                Watchers:
                3 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: