Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1832

Well support for self-describing data formats

    Details

    • Type: New Feature
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.0
    • Component/s: Planner/Optimizer
    • Labels:
      None

      Description

      Problem
      Tajo already has a support for self-describing data formats like JSON, Parquet, or ORC. While they are capable of providing schema information by themselves, users must define schema to query on them with the current implementation. To solve this inconvenience, we have to improve our query planner to support self-describing data formats well.

      Solution
      First, we need to allow omitting schema definition for the create table statement. When a query is submitted for a self-describing table, the columns which don't exist in that table will be filled with Nulls.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user jihoonson opened a pull request:

          https://github.com/apache/tajo/pull/756

          TAJO-1832: Well support for self-describing data formats

          This is still on-going work, but mostly finished.
          Honestly, I think it will be finished if I improve unit tests and cleanup some code lines.
          Before finishing it, I'd like to share my approach for this issue.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/jihoonson/tajo-2 TAJO-1832

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/tajo/pull/756.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #756


          commit 2445c7d6c4d5b6cead7db09936d08945da043acf
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-10T07:54:07Z

          TAJO-1833

          commit 9b978900eade3d6d71f9a47c405b0eab4cdb275a
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-10T08:30:19Z

          Change to abstract class

          commit 11c089f0a0c71f1cb60ab2dbb71e7769d42ff6bc
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-10T09:26:44Z

          TAJO-1832

          commit 67497269c4e248dd4430bdbf745c91e6078412dd
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-10T12:58:51Z

          TAJO-1832

          commit 01b4d92df72171437cf8cf5ee96394112d88169d
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-10T14:36:29Z

          Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1832

          commit d0be6b31cd8c3747c0b186ea35d69c6dbebe0c34
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-11T17:30:38Z

          TAJO-1832

          commit 9b58e686b207f47c645be5f519671511af84dd24
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-12T06:13:12Z

          fix groupby

          commit af8a8e4abcce248359dd95e2e06ee1b926becd92
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-12T06:50:10Z

          add license

          commit 471a70a5e628023509c5eb867660714b5e61a388
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-12T09:37:59Z

          Finished, but need to improve test

          commit 4f5ac455d21c7a9bfc13280a60cc3354010c9145
          Author: Jihoon Son <jihoonson@apache.org>
          Date: 2015-09-12T09:43:32Z

          Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1832


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user jihoonson opened a pull request: https://github.com/apache/tajo/pull/756 TAJO-1832 : Well support for self-describing data formats This is still on-going work, but mostly finished. Honestly, I think it will be finished if I improve unit tests and cleanup some code lines. Before finishing it, I'd like to share my approach for this issue. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jihoonson/tajo-2 TAJO-1832 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/756.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #756 commit 2445c7d6c4d5b6cead7db09936d08945da043acf Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-10T07:54:07Z TAJO-1833 commit 9b978900eade3d6d71f9a47c405b0eab4cdb275a Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-10T08:30:19Z Change to abstract class commit 11c089f0a0c71f1cb60ab2dbb71e7769d42ff6bc Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-10T09:26:44Z TAJO-1832 commit 67497269c4e248dd4430bdbf745c91e6078412dd Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-10T12:58:51Z TAJO-1832 commit 01b4d92df72171437cf8cf5ee96394112d88169d Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-10T14:36:29Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1832 commit d0be6b31cd8c3747c0b186ea35d69c6dbebe0c34 Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-11T17:30:38Z TAJO-1832 commit 9b58e686b207f47c645be5f519671511af84dd24 Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-12T06:13:12Z fix groupby commit af8a8e4abcce248359dd95e2e06ee1b926becd92 Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-12T06:50:10Z add license commit 471a70a5e628023509c5eb867660714b5e61a388 Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-12T09:37:59Z Finished, but need to improve test commit 4f5ac455d21c7a9bfc13280a60cc3354010c9145 Author: Jihoon Son <jihoonson@apache.org> Date: 2015-09-12T09:43:32Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1832
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-139840581

          This test failure is due to #757. Please review it first.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-139840581 This test failure is due to #757. Please review it first.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-140019030

          This patch is ready for review.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-140019030 This patch is ready for review.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39815028

          — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java —
          @@ -35,34 +36,43 @@

          public class TableDesc implements ProtoObject<TableDescProto>, GsonObject, Cloneable {
          @Expose protected String tableName; // required

          • @Expose protected Schema schema;
            + @Expose protected Schema schema; // optional for self-describing tables
            @Expose protected TableMeta meta; // required
            /** uri is set if external flag is TRUE. */
          • @Expose protected URI uri; // optional
            + @Expose protected URI uri; // optional
              • End diff –

          In the last work, I changed URI to be required. Regardless of whether it is manged or external, every TableDescs has the table URI. The evidence is unused ``TableDesc::hasUri()``, which can be removed. Additional evidence is that there is no nullity check around ``getUri()``.

          Otherwise, it is a bug.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39815028 — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java — @@ -35,34 +36,43 @@ public class TableDesc implements ProtoObject<TableDescProto>, GsonObject, Cloneable { @Expose protected String tableName; // required @Expose protected Schema schema; + @Expose protected Schema schema; // optional for self-describing tables @Expose protected TableMeta meta; // required /** uri is set if external flag is TRUE. */ @Expose protected URI uri; // optional + @Expose protected URI uri; // optional End diff – In the last work, I changed URI to be required. Regardless of whether it is manged or external, every TableDescs has the table URI. The evidence is unused ``TableDesc::hasUri()``, which can be removed. Additional evidence is that there is no nullity check around ``getUri()``. Otherwise, it is a bug.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39815240

          — Diff: tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto —
          @@ -98,10 +98,11 @@ message TableDescProto {
          required string table_name = 2;
          optional string path = 3;
          required TableProto meta = 4;

          • required SchemaProto schema = 5;
            + optional SchemaProto schema = 5;
              • End diff –

          If we can ensure that the TableDescs of all normal tables have Schema, we can use its ``optional`` constraint to indicate either self-describing or not.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39815240 — Diff: tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto — @@ -98,10 +98,11 @@ message TableDescProto { required string table_name = 2; optional string path = 3; required TableProto meta = 4; required SchemaProto schema = 5; + optional SchemaProto schema = 5; End diff – If we can ensure that the TableDescs of all normal tables have Schema, we can use its ``optional`` constraint to indicate either self-describing or not.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39816142

          — Diff: tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/RpcClientManager.java —
          @@ -199,7 +199,6 @@ public static void close() {

          • After it is shutdown it is not possible to reuse it again.
            */
            public static void shutdown() {
          • close();
              • End diff –

          Is it relevant in this patch? If so, could you explain the behind reason?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39816142 — Diff: tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/RpcClientManager.java — @@ -199,7 +199,6 @@ public static void close() { After it is shutdown it is not possible to reuse it again. */ public static void shutdown() { close(); End diff – Is it relevant in this patch? If so, could you explain the behind reason?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39816146

          — Diff: tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4 —
          @@ -106,9 +106,9 @@ if_exists
          ;

          create_table_statement

          • : CREATE EXTERNAL TABLE (if_not_exists)? table_name table_elements (TABLESPACE spacename=identifier)? USING storage_type=identifier
            + : CREATE EXTERNAL TABLE (if_not_exists)? table_name (table_elements)? (TABLESPACE spacename=identifier)? USING storage_type=identifier
              • End diff –

          This grammar allows to create table without parenthesis ``(..)``. It looks good to me.

          I also have another suggestion. How do you think about a special notation with parenthesis like ``CREATE EXTERNAL TABLE USING JSON``. I used asterisk ``*`` as a meaning of becoming anything and the infinite number of fields, which can be zero to up to be infinite.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39816146 — Diff: tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4 — @@ -106,9 +106,9 @@ if_exists ; create_table_statement : CREATE EXTERNAL TABLE (if_not_exists)? table_name table_elements (TABLESPACE spacename=identifier)? USING storage_type=identifier + : CREATE EXTERNAL TABLE (if_not_exists)? table_name (table_elements)? (TABLESPACE spacename=identifier)? USING storage_type=identifier End diff – This grammar allows to create table without parenthesis ``(..)``. It looks good to me. I also have another suggestion. How do you think about a special notation with parenthesis like ``CREATE EXTERNAL TABLE USING JSON``. I used asterisk ``*`` as a meaning of becoming anything and the infinite number of fields, which can be zero to up to be infinite.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39816462

          — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java —
          @@ -0,0 +1,479 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.plan.rewrite;
          +
          +import com.google.common.base.Objects;
          +import org.apache.tajo.SessionVars;
          +import org.apache.tajo.algebra.*;
          +import org.apache.tajo.catalog.*;
          +import org.apache.tajo.common.TajoDataTypes.Type;
          +import org.apache.tajo.exception.TajoException;
          +import org.apache.tajo.exception.TajoInternalError;
          +import org.apache.tajo.exception.UnsupportedException;
          +import org.apache.tajo.plan.ExprAnnotator;
          +import org.apache.tajo.plan.LogicalPlan;
          +import org.apache.tajo.plan.LogicalPlan.QueryBlock;
          +import org.apache.tajo.plan.LogicalPlanner.PlanContext;
          +import org.apache.tajo.plan.algebra.BaseAlgebraVisitor;
          +import org.apache.tajo.plan.logical.*;
          +import org.apache.tajo.plan.nameresolver.NameResolver;
          +import org.apache.tajo.plan.nameresolver.NameResolvingMode;
          +import org.apache.tajo.plan.rewrite.BaseSchemaBuildPhase.Processor.NameRefInSelectListNormalizer;
          +import org.apache.tajo.plan.util.ExprFinder;
          +import org.apache.tajo.plan.util.PlannerUtil;
          +import org.apache.tajo.plan.visitor.SimpleAlgebraVisitor;
          +import org.apache.tajo.util.StringUtils;
          +import org.apache.tajo.util.TUtil;
          +import org.apache.tajo.util.graph.DirectedGraphVisitor;
          +import org.apache.tajo.util.graph.SimpleDirectedGraph;
          +
          +import java.util.*;
          +
          +/**
          + * SelfDescSchemaBuildPhase builds the schema information of tables of self-describing data formats,
          + * such as JSON, Parquet, and ORC.
          + */
          +public class SelfDescSchemaBuildPhase extends LogicalPlanPreprocessPhase {
          +
          + private Processor processor;
          +
          + public SelfDescSchemaBuildPhase(CatalogService catalog, ExprAnnotator annotator)

          { + super(catalog, annotator); + }

          +
          + @Override
          + public String getName()

          { + return "Self-describing schema build phase"; + }

          +
          + private static String getQualifiedRelationName(PlanContext context, Relation relation)

          { + return CatalogUtil.isFQTableName(relation.getName()) ? + relation.getName() : + CatalogUtil.buildFQName(context.getQueryContext().get(SessionVars.CURRENT_DATABASE), relation.getName()); + }

          +
          + @Override
          + public boolean isEligible(PlanContext context, Expr expr) throws TajoException {
          + Set<Relation> relations = ExprFinderIncludeSubquery.finds(expr, OpType.Relation);
          + for (Relation eachRelation : relations) {
          + if (catalog.getTableDesc(getQualifiedRelationName(context, eachRelation)).hasSelfDescSchema())

          { + return true; + }

          + }
          + return false;
          + }
          +
          + static class FinderContext<T> {
          + Set<T> set = new HashSet<>();
          + OpType targetType;
          +
          + FinderContext(OpType type)

          { + this.targetType = type; + }

          + }
          +
          + private static class ExprFinderIncludeSubquery extends SimpleAlgebraVisitor<FinderContext, Object> {
          +
          + public static <T extends Expr> Set<T> finds(Expr expr, OpType type) throws TajoException

          { + FinderContext<T> context = new FinderContext<>(type); + ExprFinderIncludeSubquery finder = new ExprFinderIncludeSubquery(); + finder.visit(context, new Stack<Expr>(), expr); + return context.set; + }

          +
          + @Override
          + public Object visit(FinderContext ctx, Stack<Expr> stack, Expr expr) throws TajoException {
          + if (expr instanceof Selection)

          { + preHook(ctx, stack, expr); + visit(ctx, stack, ((Selection) expr).getQual()); + visitUnaryOperator(ctx, stack, (UnaryOperator) expr); + postHook(ctx, stack, expr, null); + }

          else if (expr instanceof UnaryOperator)

          { + preHook(ctx, stack, expr); + visitUnaryOperator(ctx, stack, (UnaryOperator) expr); + postHook(ctx, stack, expr, null); + }

          else if (expr instanceof BinaryOperator)

          { + preHook(ctx, stack, expr); + visitBinaryOperator(ctx, stack, (BinaryOperator) expr); + postHook(ctx, stack, expr, null); + }

          else if (expr instanceof SimpleTableSubquery)

          { + preHook(ctx, stack, expr); + visit(ctx, stack, ((SimpleTableSubquery) expr).getSubQuery()); + postHook(ctx, stack, expr, null); + }

          else if (expr instanceof TablePrimarySubQuery)

          { + preHook(ctx, stack, expr); + visit(ctx, stack, ((TablePrimarySubQuery) expr).getSubQuery()); + postHook(ctx, stack, expr, null); + }

          else

          { + super.visit(ctx, stack, expr); + }

          +
          + if (expr != null && ctx.targetType == expr.getType())

          { + ctx.set.add(expr); + }

          +
          + return null;
          + }
          + }
          +
          + @Override
          + public LogicalNode process(PlanContext context, Expr expr) throws TajoException {
          + if (processor == null)

          { + processor = new Processor(); + }

          + return processor.visit(new ProcessorContext(context), new Stack<Expr>(), expr);
          + }
          +
          + static class ProcessorContext {
          + final PlanContext planContext;
          + final Map<String, List<ColumnReferenceExpr>> projectColumns = new HashMap<>();
          +
          + public ProcessorContext(PlanContext planContext)

          { + this.planContext = planContext; + }

          + }
          +
          + static class Processor extends BaseAlgebraVisitor<ProcessorContext, LogicalNode> {
          +
          + private static <T extends LogicalNode> T getNodeFromExpr(LogicalPlan plan, Expr expr)

          { + return plan.getBlockByExpr(expr).getNodeFromExpr(expr); + }

          +
          + private static <T extends LogicalNode> T getNonRelationListExpr(LogicalPlan plan, Expr expr) {
          + if (expr instanceof RelationList)

          { + return getNodeFromExpr(plan, ((RelationList) expr).getRelations()[0]); + }

          else

          { + return getNodeFromExpr(plan, expr); + }

          + }
          +
          + @Override
          + public LogicalNode visitProjection(ProcessorContext ctx, Stack<Expr> stack, Projection expr) throws TajoException {
          + if (PlannerUtil.hasAsterisk(expr.getNamedExprs()))

          { + throw new UnsupportedException("Asterisk for self-describing data formats"); + }

          +
          + for (NamedExpr eachNamedExpr : expr.getNamedExprs()) {
          + Set<ColumnReferenceExpr> columns = ExprFinder.finds(eachNamedExpr, OpType.Column);
          + for (ColumnReferenceExpr col : columns)

          { + TUtil.putToNestedList(ctx.projectColumns, col.getQualifier(), col); + }

          + }
          +
          + super.visitProjection(ctx, stack, expr);
          +
          + ProjectionNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr);
          + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild());
          + node.setInSchema(child.getOutSchema());
          +
          + return node;
          + }
          +
          + @Override
          + public LogicalNode visitLimit(ProcessorContext ctx, Stack<Expr> stack, Limit expr) throws TajoException

          { + super.visitLimit(ctx, stack, expr); + + LimitNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitSort(ProcessorContext ctx, Stack<Expr> stack, Sort expr) throws TajoException

          { + super.visitSort(ctx, stack, expr); + + SortNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitHaving(ProcessorContext ctx, Stack<Expr> stack, Having expr) throws TajoException

          { + super.visitHaving(ctx, stack, expr); + + HavingNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitGroupBy(ProcessorContext ctx, Stack<Expr> stack, Aggregation expr) throws TajoException

          { + super.visitGroupBy(ctx, stack, expr); + + GroupbyNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitJoin(ProcessorContext ctx, Stack<Expr> stack, Join expr) throws TajoException

          { + super.visitJoin(ctx, stack, expr); + + JoinNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode leftChild = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + LogicalNode rightChild = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getRight()); + node.setInSchema(SchemaUtil.merge(leftChild.getOutSchema(), rightChild.getOutSchema())); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitFilter(ProcessorContext ctx, Stack<Expr> stack, Selection expr) throws TajoException {
          + Set<ColumnReferenceExpr> columnSet = ExprFinder.finds(expr.getQual(), OpType.Column);
          + for (ColumnReferenceExpr col : columnSet)

          { + NameRefInSelectListNormalizer.normalize(ctx.planContext, col); + TUtil.putToNestedList(ctx.projectColumns, col.getQualifier(), col); + }

          +
          + super.visitFilter(ctx, stack, expr);
          +
          + SelectionNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr);
          + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild());
          + node.setInSchema(child.getOutSchema());
          + node.setOutSchema(node.getInSchema());
          + return node;
          + }
          +
          + @Override
          + public LogicalNode visitUnion(ProcessorContext ctx, Stack<Expr> stack, SetOperation expr) throws TajoException

          { + super.visitUnion(ctx, stack, expr); + + UnionNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitExcept(ProcessorContext ctx, Stack<Expr> stack, SetOperation expr) throws TajoException

          { + super.visitExcept(ctx, stack, expr); + + ExceptNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitIntersect(ProcessorContext ctx, Stack<Expr> stack, SetOperation expr) throws TajoException

          { + super.visitIntersect(ctx, stack, expr); + + IntersectNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitSimpleTableSubquery(ProcessorContext ctx, Stack<Expr> stack, SimpleTableSubquery expr)
          + throws TajoException

          { + super.visitSimpleTableSubquery(ctx, stack, expr); + + TableSubQueryNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitTableSubQuery(ProcessorContext ctx, Stack<Expr> stack, TablePrimarySubQuery expr)
          + throws TajoException

          { + super.visitTableSubQuery(ctx, stack, expr); + + TableSubQueryNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitCreateTable(ProcessorContext ctx, Stack<Expr> stack, CreateTable expr) throws TajoException {
          + super.visitCreateTable(ctx, stack, expr);
          + CreateTableNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr);
          +
          + if (expr.hasSubQuery())

          { + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + }

          + return node;
          + }
          +
          + @Override
          + public LogicalNode visitInsert(ProcessorContext ctx, Stack<Expr> stack, Insert expr) throws TajoException

          { + super.visitInsert(ctx, stack, expr); + + InsertNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + }

          +
          + @Override
          + public LogicalNode visitRelation(ProcessorContext ctx, Stack<Expr> stack, Relation expr) throws TajoException {
          + LogicalPlan plan = ctx.planContext.getPlan();
          + QueryBlock queryBlock = plan.getBlockByExpr(expr);
          + ScanNode scan = queryBlock.getNodeFromExpr(expr);
          + TableDesc desc = scan.getTableDesc();
          +
          + if (desc.hasSelfDescSchema()) {
          + if (ctx.projectColumns.containsKey(getQualifiedRelationName(ctx.planContext, expr))) {
          + Set<Column> columns = new HashSet<>();
          + for (ColumnReferenceExpr col : ctx.projectColumns.get(getQualifiedRelationName(ctx.planContext, expr)))

          { + columns.add(NameResolver.resolve(plan, queryBlock, col, NameResolvingMode.RELS_ONLY, true)); + }

          +
          + desc.setSchema(buildSchemaFromColumnSet(columns));
          + scan.init(desc);
          + } else

          { + // error + throw new TajoInternalError( + "Columns projected from " + getQualifiedRelationName(ctx.planContext, expr) + " is not found."); + }

          + }
          +
          + return scan;
          + }
          +
          + /**
          + * This method creates a schema from a set of columns.
          + * For a nested column, its ancestors are guessed and added to the schema.
          + * For example, given a column 'glossary.title', the columns of (glossary RECORD (title TEXT)) will be added
          + * to the schema.
          + *
          + * @param columns a set of columns
          + * @return schema build from columns
          + */
          + private Schema buildSchemaFromColumnSet(Set<Column> columns) {
          + SchemaGraph schemaGraph = new SchemaGraph();
          + Set<ColumnVertex> rootVertexes = new HashSet<>();
          + Schema schema = new Schema();
          +
          + for (Column eachColumn : columns) {
          + String simpleName = eachColumn.getSimpleName();
          + if (NestedPathUtil.isPath(simpleName)) {
          + String[] paths = simpleName.split(NestedPathUtil.PATH_DELIMITER);
          + for (int i = 0; i < paths.length-1; i++) {
          + String parentName = paths[i];
          + if (i == 0)

          { + parentName = CatalogUtil.buildFQName(eachColumn.getQualifier(), parentName); + }

          + // Leaf column type is TEXT; otherwise, RECORD.
          + Type childDataType = (i == paths.length-2) ? Type.TEXT : Type.RECORD;
          — End diff –

          These lines exceed 120 column width. In addition, ``visitCreateTable()`` in this file exceeds 120 too.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39816462 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java — @@ -0,0 +1,479 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.plan.rewrite; + +import com.google.common.base.Objects; +import org.apache.tajo.SessionVars; +import org.apache.tajo.algebra.*; +import org.apache.tajo.catalog.*; +import org.apache.tajo.common.TajoDataTypes.Type; +import org.apache.tajo.exception.TajoException; +import org.apache.tajo.exception.TajoInternalError; +import org.apache.tajo.exception.UnsupportedException; +import org.apache.tajo.plan.ExprAnnotator; +import org.apache.tajo.plan.LogicalPlan; +import org.apache.tajo.plan.LogicalPlan.QueryBlock; +import org.apache.tajo.plan.LogicalPlanner.PlanContext; +import org.apache.tajo.plan.algebra.BaseAlgebraVisitor; +import org.apache.tajo.plan.logical.*; +import org.apache.tajo.plan.nameresolver.NameResolver; +import org.apache.tajo.plan.nameresolver.NameResolvingMode; +import org.apache.tajo.plan.rewrite.BaseSchemaBuildPhase.Processor.NameRefInSelectListNormalizer; +import org.apache.tajo.plan.util.ExprFinder; +import org.apache.tajo.plan.util.PlannerUtil; +import org.apache.tajo.plan.visitor.SimpleAlgebraVisitor; +import org.apache.tajo.util.StringUtils; +import org.apache.tajo.util.TUtil; +import org.apache.tajo.util.graph.DirectedGraphVisitor; +import org.apache.tajo.util.graph.SimpleDirectedGraph; + +import java.util.*; + +/** + * SelfDescSchemaBuildPhase builds the schema information of tables of self-describing data formats, + * such as JSON, Parquet, and ORC. + */ +public class SelfDescSchemaBuildPhase extends LogicalPlanPreprocessPhase { + + private Processor processor; + + public SelfDescSchemaBuildPhase(CatalogService catalog, ExprAnnotator annotator) { + super(catalog, annotator); + } + + @Override + public String getName() { + return "Self-describing schema build phase"; + } + + private static String getQualifiedRelationName(PlanContext context, Relation relation) { + return CatalogUtil.isFQTableName(relation.getName()) ? + relation.getName() : + CatalogUtil.buildFQName(context.getQueryContext().get(SessionVars.CURRENT_DATABASE), relation.getName()); + } + + @Override + public boolean isEligible(PlanContext context, Expr expr) throws TajoException { + Set<Relation> relations = ExprFinderIncludeSubquery.finds(expr, OpType.Relation); + for (Relation eachRelation : relations) { + if (catalog.getTableDesc(getQualifiedRelationName(context, eachRelation)).hasSelfDescSchema()) { + return true; + } + } + return false; + } + + static class FinderContext<T> { + Set<T> set = new HashSet<>(); + OpType targetType; + + FinderContext(OpType type) { + this.targetType = type; + } + } + + private static class ExprFinderIncludeSubquery extends SimpleAlgebraVisitor<FinderContext, Object> { + + public static <T extends Expr> Set<T> finds(Expr expr, OpType type) throws TajoException { + FinderContext<T> context = new FinderContext<>(type); + ExprFinderIncludeSubquery finder = new ExprFinderIncludeSubquery(); + finder.visit(context, new Stack<Expr>(), expr); + return context.set; + } + + @Override + public Object visit(FinderContext ctx, Stack<Expr> stack, Expr expr) throws TajoException { + if (expr instanceof Selection) { + preHook(ctx, stack, expr); + visit(ctx, stack, ((Selection) expr).getQual()); + visitUnaryOperator(ctx, stack, (UnaryOperator) expr); + postHook(ctx, stack, expr, null); + } else if (expr instanceof UnaryOperator) { + preHook(ctx, stack, expr); + visitUnaryOperator(ctx, stack, (UnaryOperator) expr); + postHook(ctx, stack, expr, null); + } else if (expr instanceof BinaryOperator) { + preHook(ctx, stack, expr); + visitBinaryOperator(ctx, stack, (BinaryOperator) expr); + postHook(ctx, stack, expr, null); + } else if (expr instanceof SimpleTableSubquery) { + preHook(ctx, stack, expr); + visit(ctx, stack, ((SimpleTableSubquery) expr).getSubQuery()); + postHook(ctx, stack, expr, null); + } else if (expr instanceof TablePrimarySubQuery) { + preHook(ctx, stack, expr); + visit(ctx, stack, ((TablePrimarySubQuery) expr).getSubQuery()); + postHook(ctx, stack, expr, null); + } else { + super.visit(ctx, stack, expr); + } + + if (expr != null && ctx.targetType == expr.getType()) { + ctx.set.add(expr); + } + + return null; + } + } + + @Override + public LogicalNode process(PlanContext context, Expr expr) throws TajoException { + if (processor == null) { + processor = new Processor(); + } + return processor.visit(new ProcessorContext(context), new Stack<Expr>(), expr); + } + + static class ProcessorContext { + final PlanContext planContext; + final Map<String, List<ColumnReferenceExpr>> projectColumns = new HashMap<>(); + + public ProcessorContext(PlanContext planContext) { + this.planContext = planContext; + } + } + + static class Processor extends BaseAlgebraVisitor<ProcessorContext, LogicalNode> { + + private static <T extends LogicalNode> T getNodeFromExpr(LogicalPlan plan, Expr expr) { + return plan.getBlockByExpr(expr).getNodeFromExpr(expr); + } + + private static <T extends LogicalNode> T getNonRelationListExpr(LogicalPlan plan, Expr expr) { + if (expr instanceof RelationList) { + return getNodeFromExpr(plan, ((RelationList) expr).getRelations()[0]); + } else { + return getNodeFromExpr(plan, expr); + } + } + + @Override + public LogicalNode visitProjection(ProcessorContext ctx, Stack<Expr> stack, Projection expr) throws TajoException { + if (PlannerUtil.hasAsterisk(expr.getNamedExprs())) { + throw new UnsupportedException("Asterisk for self-describing data formats"); + } + + for (NamedExpr eachNamedExpr : expr.getNamedExprs()) { + Set<ColumnReferenceExpr> columns = ExprFinder.finds(eachNamedExpr, OpType.Column); + for (ColumnReferenceExpr col : columns) { + TUtil.putToNestedList(ctx.projectColumns, col.getQualifier(), col); + } + } + + super.visitProjection(ctx, stack, expr); + + ProjectionNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + + return node; + } + + @Override + public LogicalNode visitLimit(ProcessorContext ctx, Stack<Expr> stack, Limit expr) throws TajoException { + super.visitLimit(ctx, stack, expr); + + LimitNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitSort(ProcessorContext ctx, Stack<Expr> stack, Sort expr) throws TajoException { + super.visitSort(ctx, stack, expr); + + SortNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitHaving(ProcessorContext ctx, Stack<Expr> stack, Having expr) throws TajoException { + super.visitHaving(ctx, stack, expr); + + HavingNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitGroupBy(ProcessorContext ctx, Stack<Expr> stack, Aggregation expr) throws TajoException { + super.visitGroupBy(ctx, stack, expr); + + GroupbyNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + return node; + } + + @Override + public LogicalNode visitJoin(ProcessorContext ctx, Stack<Expr> stack, Join expr) throws TajoException { + super.visitJoin(ctx, stack, expr); + + JoinNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode leftChild = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + LogicalNode rightChild = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getRight()); + node.setInSchema(SchemaUtil.merge(leftChild.getOutSchema(), rightChild.getOutSchema())); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitFilter(ProcessorContext ctx, Stack<Expr> stack, Selection expr) throws TajoException { + Set<ColumnReferenceExpr> columnSet = ExprFinder.finds(expr.getQual(), OpType.Column); + for (ColumnReferenceExpr col : columnSet) { + NameRefInSelectListNormalizer.normalize(ctx.planContext, col); + TUtil.putToNestedList(ctx.projectColumns, col.getQualifier(), col); + } + + super.visitFilter(ctx, stack, expr); + + SelectionNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getChild()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitUnion(ProcessorContext ctx, Stack<Expr> stack, SetOperation expr) throws TajoException { + super.visitUnion(ctx, stack, expr); + + UnionNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitExcept(ProcessorContext ctx, Stack<Expr> stack, SetOperation expr) throws TajoException { + super.visitExcept(ctx, stack, expr); + + ExceptNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitIntersect(ProcessorContext ctx, Stack<Expr> stack, SetOperation expr) throws TajoException { + super.visitIntersect(ctx, stack, expr); + + IntersectNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getLeft()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitSimpleTableSubquery(ProcessorContext ctx, Stack<Expr> stack, SimpleTableSubquery expr) + throws TajoException { + super.visitSimpleTableSubquery(ctx, stack, expr); + + TableSubQueryNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitTableSubQuery(ProcessorContext ctx, Stack<Expr> stack, TablePrimarySubQuery expr) + throws TajoException { + super.visitTableSubQuery(ctx, stack, expr); + + TableSubQueryNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitCreateTable(ProcessorContext ctx, Stack<Expr> stack, CreateTable expr) throws TajoException { + super.visitCreateTable(ctx, stack, expr); + CreateTableNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + + if (expr.hasSubQuery()) { + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + } + return node; + } + + @Override + public LogicalNode visitInsert(ProcessorContext ctx, Stack<Expr> stack, Insert expr) throws TajoException { + super.visitInsert(ctx, stack, expr); + + InsertNode node = getNodeFromExpr(ctx.planContext.getPlan(), expr); + LogicalNode child = getNonRelationListExpr(ctx.planContext.getPlan(), expr.getSubQuery()); + node.setInSchema(child.getOutSchema()); + node.setOutSchema(node.getInSchema()); + return node; + } + + @Override + public LogicalNode visitRelation(ProcessorContext ctx, Stack<Expr> stack, Relation expr) throws TajoException { + LogicalPlan plan = ctx.planContext.getPlan(); + QueryBlock queryBlock = plan.getBlockByExpr(expr); + ScanNode scan = queryBlock.getNodeFromExpr(expr); + TableDesc desc = scan.getTableDesc(); + + if (desc.hasSelfDescSchema()) { + if (ctx.projectColumns.containsKey(getQualifiedRelationName(ctx.planContext, expr))) { + Set<Column> columns = new HashSet<>(); + for (ColumnReferenceExpr col : ctx.projectColumns.get(getQualifiedRelationName(ctx.planContext, expr))) { + columns.add(NameResolver.resolve(plan, queryBlock, col, NameResolvingMode.RELS_ONLY, true)); + } + + desc.setSchema(buildSchemaFromColumnSet(columns)); + scan.init(desc); + } else { + // error + throw new TajoInternalError( + "Columns projected from " + getQualifiedRelationName(ctx.planContext, expr) + " is not found."); + } + } + + return scan; + } + + /** + * This method creates a schema from a set of columns. + * For a nested column, its ancestors are guessed and added to the schema. + * For example, given a column 'glossary.title', the columns of (glossary RECORD (title TEXT)) will be added + * to the schema. + * + * @param columns a set of columns + * @return schema build from columns + */ + private Schema buildSchemaFromColumnSet(Set<Column> columns) { + SchemaGraph schemaGraph = new SchemaGraph(); + Set<ColumnVertex> rootVertexes = new HashSet<>(); + Schema schema = new Schema(); + + for (Column eachColumn : columns) { + String simpleName = eachColumn.getSimpleName(); + if (NestedPathUtil.isPath(simpleName)) { + String[] paths = simpleName.split(NestedPathUtil.PATH_DELIMITER); + for (int i = 0; i < paths.length-1; i++) { + String parentName = paths [i] ; + if (i == 0) { + parentName = CatalogUtil.buildFQName(eachColumn.getQualifier(), parentName); + } + // Leaf column type is TEXT; otherwise, RECORD. + Type childDataType = (i == paths.length-2) ? Type.TEXT : Type.RECORD; — End diff – These lines exceed 120 column width. In addition, ``visitCreateTable()`` in this file exceeds 120 too.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39816943

          — Diff: tajo-client/src/main/proto/ClientProtos.proto —
          @@ -198,6 +198,7 @@ message CreateTableRequest {
          required TableProto meta = 4;
          required string path = 5;
          optional PartitionMethodProto partition = 6;
          + required bool hasSelfDescSchema = 7;
          — End diff –

          In this message scheme, SchemaProto is required. it may have an empty Schema? This semantic seems to be different from TableDesc. It's trivial thing. If we see more consistent semantic, the code would be more comprehensive.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39816943 — Diff: tajo-client/src/main/proto/ClientProtos.proto — @@ -198,6 +198,7 @@ message CreateTableRequest { required TableProto meta = 4; required string path = 5; optional PartitionMethodProto partition = 6; + required bool hasSelfDescSchema = 7; — End diff – In this message scheme, SchemaProto is required. it may have an empty Schema? This semantic seems to be different from TableDesc. It's trivial thing. If we see more consistent semantic, the code would be more comprehensive.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-141300294

          It is an interesting feature. I leaved comments.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-141300294 It is an interesting feature. I leaved comments.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39918694

          — Diff: tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/RpcClientManager.java —
          @@ -199,7 +199,6 @@ public static void close() {

          • After it is shutdown it is not possible to reuse it again.
            */
            public static void shutdown() {
          • close();
              • End diff –

          This is not directly related to this ticket. It is to solve the hang problem after executing any unit test.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39918694 — Diff: tajo-rpc/tajo-rpc-protobuf/src/main/java/org/apache/tajo/rpc/RpcClientManager.java — @@ -199,7 +199,6 @@ public static void close() { After it is shutdown it is not possible to reuse it again. */ public static void shutdown() { close(); End diff – This is not directly related to this ticket. It is to solve the hang problem after executing any unit test.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39918739

          — Diff: tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto —
          @@ -98,10 +98,11 @@ message TableDescProto {
          required string table_name = 2;
          optional string path = 3;
          required TableProto meta = 4;

          • required SchemaProto schema = 5;
            + optional SchemaProto schema = 5;
              • End diff –

          I agree.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39918739 — Diff: tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto — @@ -98,10 +98,11 @@ message TableDescProto { required string table_name = 2; optional string path = 3; required TableProto meta = 4; required SchemaProto schema = 5; + optional SchemaProto schema = 5; End diff – I agree.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r39918745

          — Diff: tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4 —
          @@ -106,9 +106,9 @@ if_exists
          ;

          create_table_statement

          • : CREATE EXTERNAL TABLE (if_not_exists)? table_name table_elements (TABLESPACE spacename=identifier)? USING storage_type=identifier
            + : CREATE EXTERNAL TABLE (if_not_exists)? table_name (table_elements)? (TABLESPACE spacename=identifier)? USING storage_type=identifier
              • End diff –

          Nice idea!

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r39918745 — Diff: tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4 — @@ -106,9 +106,9 @@ if_exists ; create_table_statement : CREATE EXTERNAL TABLE (if_not_exists)? table_name table_elements (TABLESPACE spacename=identifier)? USING storage_type=identifier + : CREATE EXTERNAL TABLE (if_not_exists)? table_name (table_elements)? (TABLESPACE spacename=identifier)? USING storage_type=identifier End diff – Nice idea!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-141667763

          I reflected comments.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-141667763 I reflected comments.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-142193001

          You missed to update ``<tns:base version="9">`` in ``resources/schemas//.xml`` files.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-142193001 You missed to update ``<tns:base version="9">`` in ``resources/schemas/ / .xml`` files.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r40055988

          — Diff: tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java —
          @@ -0,0 +1,185 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.engine.query;
          +
          +import org.apache.tajo.QueryTestCaseBase;
          +import org.apache.tajo.exception.AmbiguousColumnException;
          +import org.apache.tajo.exception.TajoException;
          +import org.junit.After;
          +import org.junit.Test;
          +
          +import java.io.IOException;
          +import java.sql.ResultSet;
          +import java.sql.SQLException;
          +
          +public class TestQueryOnSelfDescTable extends QueryTestCaseBase {
          +
          + public TestQueryOnSelfDescTable() throws IOException, TajoException, SQLException

          { + super(); + + executeString(String.format("create external table self_desc_table1 (*) using json location '%s'", + getDataSetFile("sample1"))).close(); + + executeString(String.format("create external table self_desc_table2 (*) using json location '%s'", + getDataSetFile("sample2"))).close(); + + executeString(String.format("create external table self_desc_table3 (*) using json location '%s'", + getDataSetFile("tweets"))).close(); + + executeString(String.format("create external table github (*) using json location '%s'", + getDataSetFile("github"))).close(); + }

          +
          + @After
          + public void teardown() throws TajoException, SQLException {
          + executeString("drop table self_desc_table1").close();
          — End diff –

          It's a trivial comment. It would be better if you put ``IF NOT EXISTS`` to them. It will suppress error messages in unexpected cases.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r40055988 — Diff: tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java — @@ -0,0 +1,185 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.engine.query; + +import org.apache.tajo.QueryTestCaseBase; +import org.apache.tajo.exception.AmbiguousColumnException; +import org.apache.tajo.exception.TajoException; +import org.junit.After; +import org.junit.Test; + +import java.io.IOException; +import java.sql.ResultSet; +import java.sql.SQLException; + +public class TestQueryOnSelfDescTable extends QueryTestCaseBase { + + public TestQueryOnSelfDescTable() throws IOException, TajoException, SQLException { + super(); + + executeString(String.format("create external table self_desc_table1 (*) using json location '%s'", + getDataSetFile("sample1"))).close(); + + executeString(String.format("create external table self_desc_table2 (*) using json location '%s'", + getDataSetFile("sample2"))).close(); + + executeString(String.format("create external table self_desc_table3 (*) using json location '%s'", + getDataSetFile("tweets"))).close(); + + executeString(String.format("create external table github (*) using json location '%s'", + getDataSetFile("github"))).close(); + } + + @After + public void teardown() throws TajoException, SQLException { + executeString("drop table self_desc_table1").close(); — End diff – It's a trivial comment. It would be better if you put ``IF NOT EXISTS`` to them. It will suppress error messages in unexpected cases.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r40056411

          — Diff: tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java —
          @@ -0,0 +1,185 @@
          +/**
          + * Licensed to the Apache Software Foundation (ASF) under one
          + * or more contributor license agreements. See the NOTICE file
          + * distributed with this work for additional information
          + * regarding copyright ownership. The ASF licenses this file
          + * to you under the Apache License, Version 2.0 (the
          + * "License"); you may not use this file except in compliance
          + * with the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +package org.apache.tajo.engine.query;
          +
          +import org.apache.tajo.QueryTestCaseBase;
          +import org.apache.tajo.exception.AmbiguousColumnException;
          +import org.apache.tajo.exception.TajoException;
          +import org.junit.After;
          +import org.junit.Test;
          +
          +import java.io.IOException;
          +import java.sql.ResultSet;
          — End diff –

          Unused import

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r40056411 — Diff: tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java — @@ -0,0 +1,185 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.engine.query; + +import org.apache.tajo.QueryTestCaseBase; +import org.apache.tajo.exception.AmbiguousColumnException; +import org.apache.tajo.exception.TajoException; +import org.junit.After; +import org.junit.Test; + +import java.io.IOException; +import java.sql.ResultSet; — End diff – Unused import
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r40056479

          — Diff: tajo-docs/src/main/sphinx/sql_language/ddl.rst —
          @@ -40,7 +40,7 @@ CREATE TABLE
          CREATE TABLE [IF NOT EXISTS] <table_name> [(<column_name> <data_type>, ... )] [TABLESPACE tablespace_name]
          [using <storage_type> [with (<key> = <value>, ...)]] [AS <select_statement>]

          • CREATE EXTERNAL TABLE [IF NOT EXISTS] <table_name> (<column_name> <data_type>, ... )
            + CREATE EXTERNAL TABLE [IF NOT EXISTS] <table_name> [(<column_name> <data_type>, ... )]
              • End diff –

          The documentation needs to reflect the changed DDL for ````.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r40056479 — Diff: tajo-docs/src/main/sphinx/sql_language/ddl.rst — @@ -40,7 +40,7 @@ CREATE TABLE CREATE TABLE [IF NOT EXISTS] <table_name> [(<column_name> <data_type>, ... )] [TABLESPACE tablespace_name] [using <storage_type> [with (<key> = <value>, ...)] ] [AS <select_statement>] CREATE EXTERNAL TABLE [IF NOT EXISTS] <table_name> (<column_name> <data_type>, ... ) + CREATE EXTERNAL TABLE [IF NOT EXISTS] <table_name> [(<column_name> <data_type>, ... )] End diff – The documentation needs to reflect the changed DDL for `` ``.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r40056554

          — Diff: tajo-docs/src/main/sphinx/table_management/json.rst —
          @@ -0,0 +1,100 @@
          +****
          +JSON
          +****
          +
          +JSON(JavaScript Object Notation) is an open standard format for data (de)serialization. Since it is simple and human-readable, it is popularly used in many fields.
          +Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them.
          +
          +============================
          +How to Create a JSON Table ?
          +============================
          +
          +You can create JSON tables using ``CREATE TABLE`` statement. (For more information, please refer to :doc:`/sql_language/ddl`.)
          — End diff –

          ``JSON tables`` should be ``a JSON table``.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r40056554 — Diff: tajo-docs/src/main/sphinx/table_management/json.rst — @@ -0,0 +1,100 @@ +**** +JSON +**** + +JSON(JavaScript Object Notation) is an open standard format for data (de)serialization. Since it is simple and human-readable, it is popularly used in many fields. +Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them. + +============================ +How to Create a JSON Table ? +============================ + +You can create JSON tables using ``CREATE TABLE`` statement. (For more information, please refer to :doc:`/sql_language/ddl`.) — End diff – ``JSON tables`` should be ``a JSON table``.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r40056595

          — Diff: tajo-docs/src/main/sphinx/table_management/json.rst —
          @@ -0,0 +1,100 @@
          +****
          +JSON
          +****
          +
          +JSON(JavaScript Object Notation) is an open standard format for data (de)serialization. Since it is simple and human-readable, it is popularly used in many fields.
          +Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them.
          +
          +============================
          +How to Create a JSON Table ?
          +============================
          +
          +You can create JSON tables using ``CREATE TABLE`` statement. (For more information, please refer to :doc:`/sql_language/ddl`.)
          +For example, please consider an example data as follows:
          +
          +.. code-block:: bash
          +
          + $ hdfs dfs -cat /table1/table.json
          + { "title" : "Hand of the King", "name" : { "first_name": "Eddard", "last_name": "Stark"}}
          + { "title" : "Assassin", "name" : { "first_name": "Arya", "last_name": "Stark"}}
          + { "title" : "Dancing Master", "name" : { "first_name": "Syrio", "last_name": "Forel"}}
          +
          +Tajo provides two alternatives to create a table for this data. First is a traditional way to create tables. Here is an example.
          — End diff –

          How about ``two ways`` instead of ``two alternatives``?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r40056595 — Diff: tajo-docs/src/main/sphinx/table_management/json.rst — @@ -0,0 +1,100 @@ +**** +JSON +**** + +JSON(JavaScript Object Notation) is an open standard format for data (de)serialization. Since it is simple and human-readable, it is popularly used in many fields. +Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them. + +============================ +How to Create a JSON Table ? +============================ + +You can create JSON tables using ``CREATE TABLE`` statement. (For more information, please refer to :doc:`/sql_language/ddl`.) +For example, please consider an example data as follows: + +.. code-block:: bash + + $ hdfs dfs -cat /table1/table.json + { "title" : "Hand of the King", "name" : { "first_name": "Eddard", "last_name": "Stark"}} + { "title" : "Assassin", "name" : { "first_name": "Arya", "last_name": "Stark"}} + { "title" : "Dancing Master", "name" : { "first_name": "Syrio", "last_name": "Forel"}} + +Tajo provides two alternatives to create a table for this data. First is a traditional way to create tables. Here is an example. — End diff – How about ``two ways`` instead of ``two alternatives``?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on a diff in the pull request:

          https://github.com/apache/tajo/pull/756#discussion_r40056645

          — Diff: tajo-docs/src/main/sphinx/table_management/json.rst —
          @@ -0,0 +1,100 @@
          +****
          +JSON
          +****
          +
          +JSON(JavaScript Object Notation) is an open standard format for data (de)serialization. Since it is simple and human-readable, it is popularly used in many fields.
          +Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them.
          +
          +============================
          +How to Create a JSON Table ?
          +============================
          +
          +You can create JSON tables using ``CREATE TABLE`` statement. (For more information, please refer to :doc:`/sql_language/ddl`.)
          +For example, please consider an example data as follows:
          +
          +.. code-block:: bash
          +
          + $ hdfs dfs -cat /table1/table.json
          + { "title" : "Hand of the King", "name" : { "first_name": "Eddard", "last_name": "Stark"}}
          + { "title" : "Assassin", "name" : { "first_name": "Arya", "last_name": "Stark"}}
          + { "title" : "Dancing Master", "name" : { "first_name": "Syrio", "last_name": "Forel"}}
          +
          +Tajo provides two alternatives to create a table for this data. First is a traditional way to create tables. Here is an example.
          +
          +.. code-block:: sql
          +
          + CREATE EXTERNAL TABLE table1 (
          + title TEXT,
          + name RECORD (
          + first_name TEXT,
          + last_name TEXT
          + )
          + ) USING JSON LOCATION '/table1/table.json';
          +
          +With this way, you need to specify every column which they want to use. This will be a tedious work, and not appropriate for flexible JSON schema.
          +Second is a simpler alternative to alleviate this problem. When you create an external table of JSON format, you can simply omit the column specification as follows:
          +
          +.. code-block:: sql
          +
          + CREATE EXTERNAL TABLE table1 USING JSON LOCATION '/table1/table.json';
          — End diff –

          It needs to reflect the changed schema description way; i.e., ````.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/756#discussion_r40056645 — Diff: tajo-docs/src/main/sphinx/table_management/json.rst — @@ -0,0 +1,100 @@ +**** +JSON +**** + +JSON(JavaScript Object Notation) is an open standard format for data (de)serialization. Since it is simple and human-readable, it is popularly used in many fields. +Tajo supports JSON as its data format. In this section, you will get an overview of how to create JSON tables and query on them. + +============================ +How to Create a JSON Table ? +============================ + +You can create JSON tables using ``CREATE TABLE`` statement. (For more information, please refer to :doc:`/sql_language/ddl`.) +For example, please consider an example data as follows: + +.. code-block:: bash + + $ hdfs dfs -cat /table1/table.json + { "title" : "Hand of the King", "name" : { "first_name": "Eddard", "last_name": "Stark"}} + { "title" : "Assassin", "name" : { "first_name": "Arya", "last_name": "Stark"}} + { "title" : "Dancing Master", "name" : { "first_name": "Syrio", "last_name": "Forel"}} + +Tajo provides two alternatives to create a table for this data. First is a traditional way to create tables. Here is an example. + +.. code-block:: sql + + CREATE EXTERNAL TABLE table1 ( + title TEXT, + name RECORD ( + first_name TEXT, + last_name TEXT + ) + ) USING JSON LOCATION '/table1/table.json'; + +With this way, you need to specify every column which they want to use. This will be a tedious work, and not appropriate for flexible JSON schema. +Second is a simpler alternative to alleviate this problem. When you create an external table of JSON format, you can simply omit the column specification as follows: + +.. code-block:: sql + + CREATE EXTERNAL TABLE table1 USING JSON LOCATION '/table1/table.json'; — End diff – It needs to reflect the changed schema description way; i.e., `` ``.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-142196312

          The patch looks nice to me. I leaved some comments.

          I have one question. Could you briefly explain the priority of name resolving in the cases where nested column names are duplicated to database and table names?

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-142196312 The patch looks nice to me. I leaved some comments. I have one question. Could you briefly explain the priority of name resolving in the cases where nested column names are duplicated to database and table names?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-142364353

          Thank you for your review.

          As the answer for your question, the ```already known information``` has a higher priority. For example, if the nested column name of a table of self-describing format has the same name with a database or a table, the database name (or the table name) has a higher priority than nested column name because it is already known information.
          However, the table has a predefined schema, NameResolver follows the existing routine even though the query involves a column of nested type.

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-142364353 Thank you for your review. As the answer for your question, the ```already known information``` has a higher priority. For example, if the nested column name of a table of self-describing format has the same name with a database or a table, the database name (or the table name) has a higher priority than nested column name because it is already known information. However, the table has a predefined schema, NameResolver follows the existing routine even though the query involves a column of nested type.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user hyunsik commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-142420825

          Excellent! The policy sounds reasonable. Here is my +1. Ship it.

          Show
          githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-142420825 Excellent! The policy sounds reasonable. Here is my +1. Ship it.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/tajo/pull/756

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/756
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Tajo-master-CODEGEN-build #531 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/531/)
          TAJO-1832: Well support for self-describing data formats. (jihoonson: rev 5a15586139c5fed29b66cbfc03b54e0bed09f5b7)

          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_3.result
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/github/github.json
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java
          • tajo-core-tests/src/test/resources/results/TestUnionQuery/testUnionAndFilter.result
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_like_1.result
          • tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/ExprAnnotator.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby2.sql
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/tweets/sample1.json
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect.sql
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestCreateTable.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolvingMode.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_1.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby3.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSort.result
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRelsAndSubExprs.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_column.result
          • tajo-docs/src/main/sphinx/table_management/text.rst
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_list.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.sql
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-client/src/main/proto/ClientProtos.proto
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby2.result
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_2.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/TypeDeterminant.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_1.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.sql
          • tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table2.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/visitor/SimpleAlgebraVisitor.java
          • tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/CreateTableNode.java
          • tajo-docs/src/main/sphinx/sql_language/ddl.rst
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testCrossJoin.result
          • tajo-plan/src/main/proto/Plan.proto
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/ExprFinder.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testCrossJoin.sql
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_2.result
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSort.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby3.result
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_2.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeSerializer.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_1.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRels.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/algebra/BaseAlgebraVisitor.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect2.sql
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample2/sample2.json
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/case_sensitivity1.result
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample1/table.json
          • tajo-core-tests/src/test/resources/dataset/TestCreateTable/json_table/table1.json
          • tajo-core-tests/src/test/resources/queries/TestCreateTable/testSelfDescTable1.sql
          • tajo-algebra/src/main/java/org/apache/tajo/algebra/CreateTable.java
          • tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table1.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect2.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverBySubExprsAndRels.java
          • tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.result
          • CHANGES
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/dictionary/AbstractTableDescriptor.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/StoreTableNode.java
          • tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java
          • tajo-docs/src/main/sphinx/table_management/file_formats.rst
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_range.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeDeserializer.java
          • tajo-docs/src/main/sphinx/table_management/json.rst
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testInSubquery.1.result
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java
          • tajo-core/src/main/java/org/apache/tajo/master/exec/CreateTableExecutor.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #531 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/531/ ) TAJO-1832 : Well support for self-describing data formats. (jihoonson: rev 5a15586139c5fed29b66cbfc03b54e0bed09f5b7) tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_3.result tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/github/github.json tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java tajo-core-tests/src/test/resources/results/TestUnionQuery/testUnionAndFilter.result tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_like_1.result tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4 tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby.sql tajo-plan/src/main/java/org/apache/tajo/plan/ExprAnnotator.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby2.sql tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/tweets/sample1.json tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestCreateTable.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolvingMode.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_1.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby3.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSort.result tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRelsAndSubExprs.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_column.result tajo-docs/src/main/sphinx/table_management/text.rst tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_list.result tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.sql tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-client/src/main/proto/ClientProtos.proto tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby2.result tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_2.result tajo-plan/src/main/java/org/apache/tajo/plan/TypeDeterminant.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_1.result tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.sql tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table2.sql tajo-plan/src/main/java/org/apache/tajo/plan/visitor/SimpleAlgebraVisitor.java tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml tajo-plan/src/main/java/org/apache/tajo/plan/logical/CreateTableNode.java tajo-docs/src/main/sphinx/sql_language/ddl.rst tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testCrossJoin.result tajo-plan/src/main/proto/Plan.proto tajo-plan/src/main/java/org/apache/tajo/plan/util/ExprFinder.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testCrossJoin.sql tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_2.result tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSort.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby3.result tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_2.result tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeSerializer.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_1.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRels.java tajo-plan/src/main/java/org/apache/tajo/plan/algebra/BaseAlgebraVisitor.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby.result tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect2.sql tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample2/sample2.json tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/case_sensitivity1.result tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample1/table.json tajo-core-tests/src/test/resources/dataset/TestCreateTable/json_table/table1.json tajo-core-tests/src/test/resources/queries/TestCreateTable/testSelfDescTable1.sql tajo-algebra/src/main/java/org/apache/tajo/algebra/CreateTable.java tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table1.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect2.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverBySubExprsAndRels.java tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.result CHANGES tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/dictionary/AbstractTableDescriptor.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.sql tajo-plan/src/main/java/org/apache/tajo/plan/logical/StoreTableNode.java tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java tajo-docs/src/main/sphinx/table_management/file_formats.rst tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_range.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.sql tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect.result tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeDeserializer.java tajo-docs/src/main/sphinx/table_management/json.rst tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testInSubquery.1.result tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java tajo-core/src/main/java/org/apache/tajo/master/exec/CreateTableExecutor.java
          Hide
          jihoonson Jihoon Son added a comment -

          Committed to master and 0.11.

          Show
          jihoonson Jihoon Son added a comment - Committed to master and 0.11.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user jihoonson commented on the pull request:

          https://github.com/apache/tajo/pull/756#issuecomment-142467922

          Thanks!

          Show
          githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/756#issuecomment-142467922 Thanks!
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Tajo-0.11.0-build #66 (See https://builds.apache.org/job/Tajo-0.11.0-build/66/)
          TAJO-1832: Well support for self-describing data formats. (jihoonson: rev a6490bddecc47be62b87cc49677036dc1352d49c)

          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_list.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRelsAndSubExprs.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/visitor/SimpleAlgebraVisitor.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby.sql
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby2.sql
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample1/table.json
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_2.result
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.result
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_range.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/TypeDeterminant.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSort.sql
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/case_sensitivity1.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSort.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/CreateTableNode.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolvingMode.java
          • tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto
          • tajo-core-tests/src/test/resources/results/TestUnionQuery/testUnionAndFilter.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/ExprAnnotator.java
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/github/github.json
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testCrossJoin.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeSerializer.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/algebra/BaseAlgebraVisitor.java
          • CHANGES
          • tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table2.sql
          • tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_1.result
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_2.result
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.sql
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testInSubquery.1.result
          • tajo-core/src/main/java/org/apache/tajo/master/exec/CreateTableExecutor.java
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_column.result
          • tajo-core-tests/src/test/resources/queries/TestCreateTable/testSelfDescTable1.sql
          • tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table1.sql
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample2/sample2.json
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRels.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_1.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/ExprFinder.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testCrossJoin.result
          • tajo-plan/src/main/proto/Plan.proto
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_3.result
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect2.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby2.result
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect2.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.result
          • tajo-docs/src/main/sphinx/table_management/json.rst
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
          • tajo-docs/src/main/sphinx/table_management/text.rst
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_1.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_2.result
          • tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeDeserializer.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverBySubExprsAndRels.java
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml
          • tajo-docs/src/main/sphinx/sql_language/ddl.rst
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_like_1.result
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
          • tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.result
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/dictionary/AbstractTableDescriptor.java
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/tweets/sample1.json
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/StoreTableNode.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby3.sql
          • tajo-algebra/src/main/java/org/apache/tajo/algebra/CreateTable.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby.result
          • tajo-core-tests/src/test/resources/dataset/TestCreateTable/json_table/table1.json
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestCreateTable.java
          • tajo-client/src/main/proto/ClientProtos.proto
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java
          • tajo-docs/src/main/sphinx/table_management/file_formats.rst
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby3.result
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.11.0-build #66 (See https://builds.apache.org/job/Tajo-0.11.0-build/66/ ) TAJO-1832 : Well support for self-describing data formats. (jihoonson: rev a6490bddecc47be62b87cc49677036dc1352d49c) tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_list.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRelsAndSubExprs.java tajo-plan/src/main/java/org/apache/tajo/plan/visitor/SimpleAlgebraVisitor.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby.sql tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby2.sql tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample1/table.json tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_2.result tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.result tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_range.result tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.sql tajo-plan/src/main/java/org/apache/tajo/plan/TypeDeterminant.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSort.sql tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/case_sensitivity1.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSort.result tajo-plan/src/main/java/org/apache/tajo/plan/logical/CreateTableNode.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolvingMode.java tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto tajo-core-tests/src/test/resources/results/TestUnionQuery/testUnionAndFilter.result tajo-plan/src/main/java/org/apache/tajo/plan/ExprAnnotator.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/github/github.json tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testCrossJoin.sql tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeSerializer.java tajo-plan/src/main/java/org/apache/tajo/plan/algebra/BaseAlgebraVisitor.java CHANGES tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table2.sql tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_1.result tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_2.result tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.sql tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testInSubquery.1.result tajo-core/src/main/java/org/apache/tajo/master/exec/CreateTableExecutor.java tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_column.result tajo-core-tests/src/test/resources/queries/TestCreateTable/testSelfDescTable1.sql tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table1.sql tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample2/sample2.json tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRels.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_1.result tajo-plan/src/main/java/org/apache/tajo/plan/util/ExprFinder.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testCrossJoin.result tajo-plan/src/main/proto/Plan.proto tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_3.result tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect2.result tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby2.result tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect2.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.result tajo-docs/src/main/sphinx/table_management/json.rst tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-docs/src/main/sphinx/table_management/text.rst tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_1.result tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_2.result tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeDeserializer.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverBySubExprsAndRels.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml tajo-docs/src/main/sphinx/sql_language/ddl.rst tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_like_1.result tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4 tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.result tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/dictionary/AbstractTableDescriptor.java tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/tweets/sample1.json tajo-plan/src/main/java/org/apache/tajo/plan/logical/StoreTableNode.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby3.sql tajo-algebra/src/main/java/org/apache/tajo/algebra/CreateTable.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby.result tajo-core-tests/src/test/resources/dataset/TestCreateTable/json_table/table1.json tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestCreateTable.java tajo-client/src/main/proto/ClientProtos.proto tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.sql tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java tajo-docs/src/main/sphinx/table_management/file_formats.rst tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby3.result
          Hide
          hudson Hudson added a comment -

          ABORTED: Integrated in Tajo-master-build #891 (See https://builds.apache.org/job/Tajo-master-build/891/)
          TAJO-1832: Well support for self-describing data formats. (jihoonson: rev 5a15586139c5fed29b66cbfc03b54e0bed09f5b7)

          • tajo-plan/src/main/java/org/apache/tajo/plan/util/ExprFinder.java
          • tajo-docs/src/main/sphinx/table_management/json.rst
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_column.result
          • tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_1.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
          • tajo-core-tests/src/test/resources/results/TestUnionQuery/testUnionAndFilter.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeSerializer.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample2/sample2.json
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby3.result
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRelsAndSubExprs.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_like_1.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_2.result
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java
          • tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table2.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/CreateTableNode.java
          • tajo-docs/src/main/sphinx/table_management/text.rst
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample1/table.json
          • tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby.result
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml
          • tajo-plan/src/main/java/org/apache/tajo/plan/TypeDeterminant.java
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/case_sensitivity1.result
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_2.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/logical/StoreTableNode.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby2.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect.sql
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby.sql
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_2.result
          • tajo-core-tests/src/test/resources/dataset/TestCreateTable/json_table/table1.json
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestCreateTable.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSort.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSort.result
          • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
          • tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table1.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRels.java
          • tajo-docs/src/main/sphinx/sql_language/ddl.rst
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_list.result
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.result
          • tajo-client/src/main/proto/ClientProtos.proto
          • tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeDeserializer.java
          • tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_1.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java
          • tajo-docs/src/main/sphinx/table_management/file_formats.rst
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.result
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testInSubquery.1.result
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby2.sql
          • tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverBySubExprsAndRels.java
          • tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/visitor/SimpleAlgebraVisitor.java
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.sql
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java
          • tajo-algebra/src/main/java/org/apache/tajo/algebra/CreateTable.java
          • tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml
          • CHANGES
          • tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby3.sql
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/github/github.json
          • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
          • tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java
          • tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/dictionary/AbstractTableDescriptor.java
          • tajo-core-tests/src/test/resources/queries/TestCreateTable/testSelfDescTable1.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect.result
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_1.result
          • tajo-plan/src/main/proto/Plan.proto
          • tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/tweets/sample1.json
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_range.result
          • tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_3.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/algebra/BaseAlgebraVisitor.java
          • tajo-core/src/main/java/org/apache/tajo/master/exec/CreateTableExecutor.java
          • tajo-plan/src/main/java/org/apache/tajo/plan/ExprAnnotator.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testCrossJoin.sql
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect2.sql
          • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testCrossJoin.result
          • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolvingMode.java
          • tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.sql
          • tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect2.result
          Show
          hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-build #891 (See https://builds.apache.org/job/Tajo-master-build/891/ ) TAJO-1832 : Well support for self-describing data formats. (jihoonson: rev 5a15586139c5fed29b66cbfc03b54e0bed09f5b7) tajo-plan/src/main/java/org/apache/tajo/plan/util/ExprFinder.java tajo-docs/src/main/sphinx/table_management/json.rst tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_column.result tajo-client/src/main/java/org/apache/tajo/client/CatalogAdminClientImpl.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_1.result tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-core-tests/src/test/resources/results/TestUnionQuery/testUnionAndFilter.result tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeSerializer.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample2/sample2.json tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby3.result tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRelsAndSubExprs.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_like_1.result tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/LogicalPlanPreprocessPhase.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_2.result tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/store/AbstractDBStore.java tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table2.sql tajo-plan/src/main/java/org/apache/tajo/plan/logical/CreateTableNode.java tajo-docs/src/main/sphinx/table_management/text.rst tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/sample1/table.json tajo-cluster-tests/src/test/java/org/apache/tajo/QueryTestCaseBase.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby.result tajo-catalog/tajo-catalog-server/src/main/resources/schemas/derby/derby.xml tajo-plan/src/main/java/org/apache/tajo/plan/TypeDeterminant.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mariadb/mariadb.xml tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/case_sensitivity1.result tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_2.result tajo-plan/src/main/java/org/apache/tajo/plan/logical/StoreTableNode.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testGroupby2.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect.sql tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby.sql tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_nested_2.result tajo-core-tests/src/test/resources/dataset/TestCreateTable/json_table/table1.json tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseSchemaBuildPhase.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestCreateTable.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSort.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSort.result tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/TableDesc.java tajo-core-tests/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-core-tests/src/test/resources/queries/TestSQLAnalyzer/positive/create_self_desc_table1.sql tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByRels.java tajo-docs/src/main/sphinx/sql_language/ddl.rst tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_list.result tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.result tajo-client/src/main/proto/ClientProtos.proto tajo-plan/src/main/java/org/apache/tajo/plan/serder/LogicalNodeDeserializer.java tajo-catalog/tajo-catalog-common/src/main/proto/CatalogProtos.proto tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/postgresql.xml tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/SelfDescSchemaBuildPhase.java tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_1.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java tajo-docs/src/main/sphinx/table_management/file_formats.rst tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.result tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestQueryOnSelfDescTable.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testInSubquery.1.result tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby2.sql tajo-core/src/main/java/org/apache/tajo/master/TajoMasterClientService.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverBySubExprsAndRels.java tajo-core/src/main/java/org/apache/tajo/parser/sql/SQLAnalyzer.java tajo-plan/src/main/java/org/apache/tajo/plan/visitor/SimpleAlgebraVisitor.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/mysql/mysql.xml tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinOfSelfDescTablesWithQualifiedColumns.sql tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable2.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java tajo-algebra/src/main/java/org/apache/tajo/algebra/CreateTable.java tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/oracle.xml CHANGES tajo-sql-parser/src/main/antlr4/org/apache/tajo/parser/sql/SQLParser.g4 tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testGroupby3.sql tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/github/github.json tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-core-tests/src/test/java/org/apache/tajo/engine/query/TestUnionQuery.java tajo-catalog/tajo-catalog-server/src/main/java/org/apache/tajo/catalog/dictionary/AbstractTableDescriptor.java tajo-core-tests/src/test/resources/queries/TestCreateTable/testSelfDescTable1.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect.result tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_hash_1.result tajo-plan/src/main/proto/Plan.proto tajo-core-tests/src/test/resources/dataset/TestQueryOnSelfDescTable/tweets/sample1.json tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_partition_by_range.result tajo-core-tests/src/test/resources/results/TestSQLAnalyzer/create_table_maptype_3.result tajo-plan/src/main/java/org/apache/tajo/plan/algebra/BaseAlgebraVisitor.java tajo-core/src/main/java/org/apache/tajo/master/exec/CreateTableExecutor.java tajo-plan/src/main/java/org/apache/tajo/plan/ExprAnnotator.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testCrossJoin.sql tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testSelect2.sql tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/BaseLogicalPlanPreprocessPhaseProvider.java tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testCrossJoin.result tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolvingMode.java tajo-core-tests/src/test/resources/queries/TestQueryOnSelfDescTable/testJoinWithSchemaFullTable4.sql tajo-core-tests/src/test/resources/results/TestQueryOnSelfDescTable/testSelect2.result

            People

            • Assignee:
              jihoonson Jihoon Son
              Reporter:
              jihoonson Jihoon Son
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development