Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1833

Refine LogicalPlanPreprocessor to add new rules easily

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      LogicalPlanPreprocessor is responsible for building base schema information. So far, it has been enough to do this.
      However, to support self-describing data formats well (TAJO-1832), the current architecture is difficult to fulfil the requirements.

      To make good use of self-described data format information, schema can be guessed at query runtime. I think that this should be done between logical plan preprocessing phase and logical planning phase. However, guessing schema is also a duty of LogicalPlanPreprocessor, that is building base schema information.

      For the sake of generality and extensibility, I think we need to refine LogicalPlanPreprocessor to add new rules easily.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user jihoonson opened a pull request:

          https://github.com/apache/tajo/pull/752

          TAJO-1833: Refine LogicalPlanPreprocessor to add new rules easily

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/jihoonson/tajo-2 TAJO-1833

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/752.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #752


          commit 2445c7d6c4d5b6cead7db09936d08945da043acf
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-10T07:54:07Z

          TAJO-1833


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user jihoonson opened a pull request: https://github.com/apache/tajo/pull/752 TAJO-1833 : Refine LogicalPlanPreprocessor to add new rules easily You can merge this pull request into a Git repository by running: $ git pull https://github.com/jihoonson/tajo-2 TAJO-1833 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/752.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #752 commit 2445c7d6c4d5b6cead7db09936d08945da043acf Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-10T07:54:07Z TAJO-1833
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/752#issuecomment-139227997

          +1
          The patch looks good to me.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/752#issuecomment-139227997 +1 The patch looks good to me.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #498 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/498/)
          TAJO-1833: Refine LogicalPlanPreprocessor to add new rules easily. (jihoonson: rev 508d17a2d7fdb3f5ac9a9dabec46b6ec4e5fb1cb)

          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • CHANGES
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhaseProvider.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteRule.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanRewriteEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteEngine.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #498 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/498/ ) TAJO-1833 : Refine LogicalPlanPreprocessor to add new rules easily. (jihoonson: rev 508d17a2d7fdb3f5ac9a9dabec46b6ec4e5fb1cb) tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java CHANGES tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhaseProvider.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteRule.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanRewriteEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteEngine.java
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/752#issuecomment-139228770

          Thanks for quick review!

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/752#issuecomment-139228770 Thanks for quick review!
          Hide
          jihoonson Jihoon Son added a comment -

          Committed to master and 0.11.

          Show
          jihoonson Jihoon Son added a comment - Committed to master and 0.11.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/752

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/752
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-0.11.0-build #33 (See https://builds.apache.org/job/Tajo-0.11.0-build/33/)
          TAJO-1833: Refine LogicalPlanPreprocessor to add new rules easily. (jihoonson: rev 357b7be815e61bc55b5de6a6161ce1d9282fdf99)

          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteRule.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhaseProvider.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanRewriteEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • CHANGES
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-0.11.0-build #33 (See https://builds.apache.org/job/Tajo-0.11.0-build/33/ ) TAJO-1833 : Refine LogicalPlanPreprocessor to add new rules easily. (jihoonson: rev 357b7be815e61bc55b5de6a6161ce1d9282fdf99) tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteRule.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhaseProvider.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanRewriteEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java CHANGES tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-master-build #856 (See https://builds.apache.org/job/Tajo-master-build/856/)
          TAJO-1833: Refine LogicalPlanPreprocessor to add new rules easily. (jihoonson: rev 508d17a2d7fdb3f5ac9a9dabec46b6ec4e5fb1cb)

          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteRule.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessEngine.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhaseProvider.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanRewriteEngine.java
          • CHANGES
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #856 (See https://builds.apache.org/job/Tajo-master-build/856/ ) TAJO-1833 : Refine LogicalPlanPreprocessor to add new rules easily. (jihoonson: rev 508d17a2d7fdb3f5ac9a9dabec46b6ec4e5fb1cb) tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanRewriteRule.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhaseProvider.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanRewriteEngine.java CHANGES

            People

            • Assignee:
              jihoonson Jihoon Son
              Reporter:
              jihoonson Jihoon Son
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development