Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-721 Add support for nested tuple or struct type
  3. TAJO-1359

Add nested field projector and language extension to project nested record

    Details

      Description

      We need to improve Projector class to get nested record fields, and we also add some language extension to specify certain nested records in table schema. Both works should be done together. Otherwise, we need to test an entire work process.

      Using dot '.' would be good for the syntax to specify nested fields. Many systems (Hive, Google BigQuery, and Drill) already use this syntax. Probably, many users are familiar with this form.

      For example, if employee is a root nested record field and it includes age and name fields, consisting two fields lastname and firstname, we can specify them individually as follows:

      SELECT employee.age, employee.name.lastname, employee.name.firstname FROM ...
      
      1. TAJO-1359_2.patch
        159 kB
        Hyunsik Choi
      2. TAJO-1359_3.patch
        160 kB
        Hyunsik Choi
      3. TAJO-1359_4.patch
        161 kB
        Hyunsik Choi
      4. TAJO-1359_5.patch
        164 kB
        Hyunsik Choi
      5. TAJO-1359_6.patch
        168 kB
        Hyunsik Choi
      6. TAJO-1359_7.patch
        168 kB
        Hyunsik Choi
      7. TAJO-1359_8.patch
        173 kB
        Hyunsik Choi
      8. TAJO-1359_9.patch
        175 kB
        Hyunsik Choi
      9. TAJO-1359.patch
        150 kB
        Hyunsik Choi

        Activity

        Hide
        hyunsik Hyunsik Choi added a comment -

        This work involves some changes of NameResolver and some logics to recognize nested fields.

        Show
        hyunsik Hyunsik Choi added a comment - This work involves some changes of NameResolver and some logics to recognize nested fields.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I almost finish this work. I'll submit the patch soon. This patch supports JSON as the first nested record supported file type. Then, I'll start TAJO-1363 and TAJO-1364.

        Show
        hyunsik Hyunsik Choi added a comment - I almost finish this work. I'll submit the patch soon. This patch supports JSON as the first nested record supported file type. Then, I'll start TAJO-1363 and TAJO-1364 .
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user hyunsik opened a pull request:

        https://github.com/apache/tajo/pull/422

        TAJO-1359: Add nested field projector and language extension to project nested record.

        This work is still on going. Actually, all features look well. But, I need to clean up and refactor the changes, and I'll add more unite tests to verify more use cases of complex types.

        For tests, I'm using a twitter example data at https://dev.twitter.com/rest/reference/get/search/tweets. Projecting all fields including nested primitive fields work well. I'm going to add deeper nested schema and complex cases.

        Here is an example DDL and a query statement.
        DDL
        ```sql
        CREATE EXTERNAL TABLE tweets (
        coordinates TEXT,
        favorited BOOL,
        truncated BOOL,
        created_at TIMESTAMP,
        id_str TEXT,
        /*entrities RECORD (
        urls ARRAY<TEXT>
        )*/
        in_reply_to_user_id_str TEXT,
        contributors TEXT,
        text TEXT,
        metadata RECORD (
        iso_language_code TEXT,
        result_type TEXT
        ),
        retweet_count INTEGER,
        in_reply_to_status_id_str TEXT,
        id TEXT,
        geo TEXT,
        retweeted BOOL,
        in_reply_to_user_id TEXT,
        place TEXT,
        user RECORD (
        profile_sidebar_fill_color TEXT,
        profile_sidebar_border_color TEXT,
        profile_background_tile TEXT,
        name TEXT,
        profile_image_url TEXT,
        created_at TIMESTAMP,
        location TEXT,
        follow_request_sent TEXT,
        profile_link_color TEXT,
        is_translator BOOL,
        id_str TEXT,
        /*
        entities RECORD (
        url RECORD (
        ),
        description RECORD (
        )
        ), */
        default_profile BOOL,
        contributors_enabled BOOL,
        favourites_count INTEGER,
        url TEXT,
        profile_image_url_https TEXT,
        utc_offset INTEGER,
        id BIGINT,
        profile_use_background_image BOOL,
        listed_count INTEGER,
        profile_text_color TEXT,
        lang TEXT,
        followers_count INTEGER,
        protected BOOL,
        notifications TEXT,
        profile_background_image_url_https TEXT,
        profile_background_color TEXT,
        verified TEXT,
        geo_enabled TEXT,
        time_zone TEXT,
        description TEXT,
        default_profile_image TEXT,
        profile_background_image_url TEXT,
        statuses_count INTEGER,
        friends_count INTEGER,
        following TEXT,
        show_all_inline_media BOOL,
        screen_name TEXT
        ),
        in_reply_to_screen_name TEXT,
        source TEXT,
        in_reply_to_status_id TEXT
        ) USING JSON LOCATION $

        {table.path}

        ;
        ```

        DML
        ```sql
        SELECT
        coordinates,
        favorited,
        truncated,
        created_at,
        id_str,
        in_reply_to_user_id_str,
        contributors,
        "text",
        metadata.iso_language_code,
        metadata.result_type,
        retweet_count,
        in_reply_to_status_id_str,
        id,
        geo,
        retweeted,
        in_reply_to_user_id,
        place,
        user.profile_sidebar_fill_color,
        user.profile_sidebar_border_color,
        user.profile_background_tile,
        user.name,
        user.profile_image_url,
        user.created_at,
        user.location,
        user.follow_request_sent,
        user.profile_link_color,
        user.is_translator,
        user.id_str,
        user.default_profile,
        user.contributors_enabled,
        user.favourites_count,
        user.url,
        user.profile_image_url_https,
        user.utc_offset,
        user.id,
        user.profile_use_background_image,
        user.listed_count,
        user.profile_text_color,
        user.lang,
        user.followers_count,
        user.protected,
        user.notifications,
        user.profile_background_image_url_https,
        user.profile_background_color,
        user.verified,
        user.geo_enabled,
        user.time_zone,
        user.description,
        user.default_profile_image,
        user.profile_background_image_url,
        user.statuses_count,
        user.friends_count,
        user.following,
        user.show_all_inline_media,
        user.screen_name,
        in_reply_to_screen_name,
        source,
        in_reply_to_status_id
        FROM
        tweets;
        ```

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/hyunsik/tajo TAJO-1359

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/422.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #422


        commit ea6aeb38075fb3684ec33dacdf6d7769ef6c7e5f
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-21T01:56:06Z

        TAJO-1353: CREATE TABLE should support the nested record definition.

        commit e3ad19f461739d5cc20b7d0b0cfac0f7f16ee2e5
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-21T02:27:06Z

        Changed the type name 'struct' to 'record'.

        commit bb39a913b8ac8230eb97d65858cf93f95ffd68fc
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-21T07:47:33Z

        Introduced TypeDesc which contains DataType and NestedSchema.

        commit a21c110c6a48f129d9c56c8680db5eea593d9076
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-22T23:48:15Z

        Introduced nested record to schema almostly.

        commit 1c6b21d5647d5b9e62ff7287df46fe1bd8048661
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-27T05:15:13Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329

        Conflicts:
        tajo-algebra/src/main/java/org/apache/tajo/algebra/ColumnDefinition.java
        tajo-algebra/src/main/java/org/apache/tajo/algebra/DataTypeExpr.java

        commit 1c8d2585c1a13f16fd20acc5076b9fb1f734c91c
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-27T12:10:05Z

        TAJO-1329: Improve Schema class to support nested struct support.

        commit 0ef60183c48f77bd03ac3eb9ef80d62dbe66bf0e
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-27T19:52:32Z

        Change child_fields_num to nested_field_num.

        • Add more comments.
        • Clean up some codes.

        commit 458ed0e5521101362d54c395242c9892374abfe7
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-02-28T23:47:33Z

        Refactor resolver and its releated things:

        • Refactor TUtil::collectionToString and TUtil::arrayToString to StringUtils::join.
        • Add SQLAnalyzer::visitColumn_reference to support dotted-chained identifier.
        • Clean up NameResolver.

        commit 66693e4d842d9e82a4161da62989fb10b9cd9fc9
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-05T20:22:55Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329

        commit a2d07415faa224849786954eeb47ebc7c3d90d36
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-05T21:08:10Z

        Allow catalog stores to have duplicated names across the different levels.

        commit 8c000ce9ee53c5547f80f6ad412b5f2a560e1e37
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-06T06:45:31Z

        Fixed unique key of derby.

        commit 4ca4e0e0884c5aa51155a8a5a52b92783cc755b8
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-11T09:15:14Z

        In progress.

        commit 50076d89b4902650136a6b6c5a4a256b12d6a7bb
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-11T09:15:23Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359

        commit 38c61e14cb8fde17cb9cd1c96fe6112ecaf8f9ac
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-11T09:16:04Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329

        commit f5fdb02c9a1b0df9cddb5f783c21815d6522ec26
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-11T10:03:11Z

        Add missed nested field support.

        • Improve tajo-dump to support nested schema
        • enable \d command to show nested schema

        commit c2521114e0efb33cdb78ab92b045f519612e92d2
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-11T10:04:40Z

        Merge branch 'TAJO-1329' of github.com:hyunsik/tajo into TAJO-1329

        commit cc5ab77a9b07f53b265b25fddbf30e4d5ec08c81
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-11T10:40:55Z

        Merge branch 'TAJO-1329' of github.com:hyunsik/tajo into TAJO-1359

        commit 8b3f45ed8aac792fc79de3a86a529ac4c1e8ee91
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-12T01:27:05Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359

        commit f10f752ebebe88eaf1137b7de9241126786a03d3
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-12T09:16:17Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329

        commit b271d4d8946a6db26a25c83db1c8893d4ee24746
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-12T19:17:44Z

        Update schema in xml files.

        commit cfa95ebd5bbae96d3f0b593283e4ca81518ee31b
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-12T19:28:43Z

        Updated catalog store driver versions.

        commit 6b4e38aea84bba893c34b031312232ad56603d2b
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-12T19:30:10Z

        Merge branch 'TAJO-1329' of github.com:hyunsik/tajo into TAJO-1359

        commit 350621221e20d5237c15adb0f135b92753a80cb2
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-13T00:20:35Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJo-1329

        Conflicts:
        tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/columns.sql
        tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/columns.sql

        commit f3cf891791fef92d2872746c22f9f64c51df64ba
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-13T01:27:58Z

        Fixed the bug about the duplicated names in different level.

        commit 2567c32f9e05479b21cbf48f52f94d4df8728ffc
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-13T02:19:48Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359

        Conflicts:
        tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/columns.sql
        tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/columns.sql

        commit 04bb0e1e1c8e58f86e923f768b711e1f3bebfe1a
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-13T02:20:18Z

        Merge branch 'TAJO-1329' of github.com:hyunsik/tajo into TAJO-1359

        Conflicts:
        tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/columns.sql
        tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/columns.sql

        commit 6a6eb1828383424df569b6b54cd27df0137dce97
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-13T06:58:59Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359

        commit 899deead496363ddcca217bacc16f9490cf3271c
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-14T02:22:21Z

        Completed complex type support.

        • Rename Schema::getColumns() to getRootColumns.
        • Add Schema::getAllColumns to return all columns flatten.
        • Change the behavior of projectable scanner.
        • Improved DelimitedTextLine scanner to return only compact projected fields.

        commit ed3b54ab686a6d47474acf22a0e74e413e7c397b
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-14T02:24:57Z

        Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359

        Conflicts:
        tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java

        commit d2bef28361817e59bd5cbcbfcfeda1c8b1a30fea
        Author: Hyunsik Choi <hyunsik@apache.org>
        Date: 2015-03-14T06:58:00Z

        Added twitter json example tests.


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user hyunsik opened a pull request: https://github.com/apache/tajo/pull/422 TAJO-1359 : Add nested field projector and language extension to project nested record. This work is still on going. Actually, all features look well. But, I need to clean up and refactor the changes, and I'll add more unite tests to verify more use cases of complex types. For tests, I'm using a twitter example data at https://dev.twitter.com/rest/reference/get/search/tweets . Projecting all fields including nested primitive fields work well. I'm going to add deeper nested schema and complex cases. Here is an example DDL and a query statement. DDL ```sql CREATE EXTERNAL TABLE tweets ( coordinates TEXT, favorited BOOL, truncated BOOL, created_at TIMESTAMP, id_str TEXT, /*entrities RECORD ( urls ARRAY<TEXT> )*/ in_reply_to_user_id_str TEXT, contributors TEXT, text TEXT, metadata RECORD ( iso_language_code TEXT, result_type TEXT ), retweet_count INTEGER, in_reply_to_status_id_str TEXT, id TEXT, geo TEXT, retweeted BOOL, in_reply_to_user_id TEXT, place TEXT, user RECORD ( profile_sidebar_fill_color TEXT, profile_sidebar_border_color TEXT, profile_background_tile TEXT, name TEXT, profile_image_url TEXT, created_at TIMESTAMP, location TEXT, follow_request_sent TEXT, profile_link_color TEXT, is_translator BOOL, id_str TEXT, /* entities RECORD ( url RECORD ( ), description RECORD ( ) ), */ default_profile BOOL, contributors_enabled BOOL, favourites_count INTEGER, url TEXT, profile_image_url_https TEXT, utc_offset INTEGER, id BIGINT, profile_use_background_image BOOL, listed_count INTEGER, profile_text_color TEXT, lang TEXT, followers_count INTEGER, protected BOOL, notifications TEXT, profile_background_image_url_https TEXT, profile_background_color TEXT, verified TEXT, geo_enabled TEXT, time_zone TEXT, description TEXT, default_profile_image TEXT, profile_background_image_url TEXT, statuses_count INTEGER, friends_count INTEGER, following TEXT, show_all_inline_media BOOL, screen_name TEXT ), in_reply_to_screen_name TEXT, source TEXT, in_reply_to_status_id TEXT ) USING JSON LOCATION $ {table.path} ; ``` DML ```sql SELECT coordinates, favorited, truncated, created_at, id_str, in_reply_to_user_id_str, contributors, "text", metadata.iso_language_code, metadata.result_type, retweet_count, in_reply_to_status_id_str, id, geo, retweeted, in_reply_to_user_id, place, user.profile_sidebar_fill_color, user.profile_sidebar_border_color, user.profile_background_tile, user.name, user.profile_image_url, user.created_at, user.location, user.follow_request_sent, user.profile_link_color, user.is_translator, user.id_str, user.default_profile, user.contributors_enabled, user.favourites_count, user.url, user.profile_image_url_https, user.utc_offset, user.id, user.profile_use_background_image, user.listed_count, user.profile_text_color, user.lang, user.followers_count, user.protected, user.notifications, user.profile_background_image_url_https, user.profile_background_color, user.verified, user.geo_enabled, user.time_zone, user.description, user.default_profile_image, user.profile_background_image_url, user.statuses_count, user.friends_count, user.following, user.show_all_inline_media, user.screen_name, in_reply_to_screen_name, source, in_reply_to_status_id FROM tweets; ``` You can merge this pull request into a Git repository by running: $ git pull https://github.com/hyunsik/tajo TAJO-1359 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/422.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #422 commit ea6aeb38075fb3684ec33dacdf6d7769ef6c7e5f Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-21T01:56:06Z TAJO-1353 : CREATE TABLE should support the nested record definition. commit e3ad19f461739d5cc20b7d0b0cfac0f7f16ee2e5 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-21T02:27:06Z Changed the type name 'struct' to 'record'. commit bb39a913b8ac8230eb97d65858cf93f95ffd68fc Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-21T07:47:33Z Introduced TypeDesc which contains DataType and NestedSchema. commit a21c110c6a48f129d9c56c8680db5eea593d9076 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-22T23:48:15Z Introduced nested record to schema almostly. commit 1c6b21d5647d5b9e62ff7287df46fe1bd8048661 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-27T05:15:13Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329 Conflicts: tajo-algebra/src/main/java/org/apache/tajo/algebra/ColumnDefinition.java tajo-algebra/src/main/java/org/apache/tajo/algebra/DataTypeExpr.java commit 1c8d2585c1a13f16fd20acc5076b9fb1f734c91c Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-27T12:10:05Z TAJO-1329 : Improve Schema class to support nested struct support. commit 0ef60183c48f77bd03ac3eb9ef80d62dbe66bf0e Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-27T19:52:32Z Change child_fields_num to nested_field_num. Add more comments. Clean up some codes. commit 458ed0e5521101362d54c395242c9892374abfe7 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-02-28T23:47:33Z Refactor resolver and its releated things: Refactor TUtil::collectionToString and TUtil::arrayToString to StringUtils::join. Add SQLAnalyzer::visitColumn_reference to support dotted-chained identifier. Clean up NameResolver. commit 66693e4d842d9e82a4161da62989fb10b9cd9fc9 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-05T20:22:55Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329 commit a2d07415faa224849786954eeb47ebc7c3d90d36 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-05T21:08:10Z Allow catalog stores to have duplicated names across the different levels. commit 8c000ce9ee53c5547f80f6ad412b5f2a560e1e37 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-06T06:45:31Z Fixed unique key of derby. commit 4ca4e0e0884c5aa51155a8a5a52b92783cc755b8 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-11T09:15:14Z In progress. commit 50076d89b4902650136a6b6c5a4a256b12d6a7bb Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-11T09:15:23Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359 commit 38c61e14cb8fde17cb9cd1c96fe6112ecaf8f9ac Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-11T09:16:04Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329 commit f5fdb02c9a1b0df9cddb5f783c21815d6522ec26 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-11T10:03:11Z Add missed nested field support. Improve tajo-dump to support nested schema enable \d command to show nested schema commit c2521114e0efb33cdb78ab92b045f519612e92d2 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-11T10:04:40Z Merge branch ' TAJO-1329 ' of github.com:hyunsik/tajo into TAJO-1329 commit cc5ab77a9b07f53b265b25fddbf30e4d5ec08c81 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-11T10:40:55Z Merge branch ' TAJO-1329 ' of github.com:hyunsik/tajo into TAJO-1359 commit 8b3f45ed8aac792fc79de3a86a529ac4c1e8ee91 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-12T01:27:05Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359 commit f10f752ebebe88eaf1137b7de9241126786a03d3 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-12T09:16:17Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1329 commit b271d4d8946a6db26a25c83db1c8893d4ee24746 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-12T19:17:44Z Update schema in xml files. commit cfa95ebd5bbae96d3f0b593283e4ca81518ee31b Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-12T19:28:43Z Updated catalog store driver versions. commit 6b4e38aea84bba893c34b031312232ad56603d2b Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-12T19:30:10Z Merge branch ' TAJO-1329 ' of github.com:hyunsik/tajo into TAJO-1359 commit 350621221e20d5237c15adb0f135b92753a80cb2 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-13T00:20:35Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJo-1329 Conflicts: tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/columns.sql commit f3cf891791fef92d2872746c22f9f64c51df64ba Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-13T01:27:58Z Fixed the bug about the duplicated names in different level. commit 2567c32f9e05479b21cbf48f52f94d4df8728ffc Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-13T02:19:48Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359 Conflicts: tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/columns.sql commit 04bb0e1e1c8e58f86e923f768b711e1f3bebfe1a Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-13T02:20:18Z Merge branch ' TAJO-1329 ' of github.com:hyunsik/tajo into TAJO-1359 Conflicts: tajo-catalog/tajo-catalog-server/src/main/resources/schemas/oracle/columns.sql tajo-catalog/tajo-catalog-server/src/main/resources/schemas/postgresql/columns.sql commit 6a6eb1828383424df569b6b54cd27df0137dce97 Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-13T06:58:59Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359 commit 899deead496363ddcca217bacc16f9490cf3271c Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-14T02:22:21Z Completed complex type support. Rename Schema::getColumns() to getRootColumns. Add Schema::getAllColumns to return all columns flatten. Change the behavior of projectable scanner. Improved DelimitedTextLine scanner to return only compact projected fields. commit ed3b54ab686a6d47474acf22a0e74e413e7c397b Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-14T02:24:57Z Merge branch 'master' of https://git-wip-us.apache.org/repos/asf/tajo into TAJO-1359 Conflicts: tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java commit d2bef28361817e59bd5cbcbfcfeda1c8b1a30fea Author: Hyunsik Choi <hyunsik@apache.org> Date: 2015-03-14T06:58:00Z Added twitter json example tests.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-80042356

        This patch also includes some refactoring of Schema class as follows:

        • Rename Schema::getColumns to Schema::getRootColumns.
        • It just returns only root columns in a schema.
        • Add Schema::getAllColumns, returning all flatten columns including nested fields.
        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-80042356 This patch also includes some refactoring of Schema class as follows: Rename Schema::getColumns to Schema::getRootColumns. It just returns only root columns in a schema. Add Schema::getAllColumns, returning all flatten columns including nested fields.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-80072105

        I used this site to convert jsons to json lines.
        http://www.freeformatter.com/json-formatter.html

        I think that it would be helpful to make test data sets.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-80072105 I used this site to convert jsons to json lines. http://www.freeformatter.com/json-formatter.html I think that it would be helpful to make test data sets.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-80105505

        Thanks for the great work!
        I'll review soon.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-80105505 Thanks for the great work! I'll review soon.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-81310658

        I looked over the above queries.
        Basically, it looks good to support the nested schema.
        I have a question.
        It looks that a record column and a table have the same name. In this case, how can we identify them exactly?
        Additionally, I have one suggestion.
        As for tables, it would be good if we support the alias names for record types. It will be helpful when the record column has a long name.
        What do you think?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-81310658 I looked over the above queries. Basically, it looks good to support the nested schema. I have a question. It looks that a record column and a table have the same name. In this case, how can we identify them exactly? Additionally, I have one suggestion. As for tables, it would be good if we support the alias names for record types. It will be helpful when the record column has a long name. What do you think?
        Hide
        hyunsik Hyunsik Choi added a comment -

        I'd like to share my progress. Its almost done. There remain still some refactoring issues. I'll finish it soon.

        Show
        hyunsik Hyunsik Choi added a comment - I'd like to share my progress. Its almost done. There remain still some refactoring issues. I'll finish it soon.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12705581/TAJO-1359.patch
        against master revision release-0.9.0-rc0-208-g725448c.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 20 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 86 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs:
        org.apache.tajo.engine.query.TestSelectNestedRecord

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/623//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-cli.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/623//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705581/TAJO-1359.patch against master revision release-0.9.0-rc0-208-g725448c. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 86 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs: org.apache.tajo.engine.query.TestSelectNestedRecord Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/623//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-cli.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/623//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/623//console This message is automatically generated.
        Hide
        hyunsik Hyunsik Choi added a comment -

        Updated the patch. This patch includes the following changes:

        • Improve Scanners except for CSVFile and SequenceFile to produce only projected columns without NullDatum.
        • It will reduce the memory consumption and object creation times.
        • It is also required to easily implement nested schema support.
        • Improve JsonScanner to use materialized types rather than calling Schema::getColumn() method.
        • Add more comments.

        This patch is ready to be review. So, please review this.

        Show
        hyunsik Hyunsik Choi added a comment - Updated the patch. This patch includes the following changes: Improve Scanners except for CSVFile and SequenceFile to produce only projected columns without NullDatum. It will reduce the memory consumption and object creation times. It is also required to easily implement nested schema support. Improve JsonScanner to use materialized types rather than calling Schema::getColumn() method. Add more comments. This patch is ready to be review. So, please review this.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-83844491

        Hi @jihoonson,

        > It looks that a record column and a table have the same name. In this case, how can we identify them exactly?

        Here is how to resolve names.
        https://github.com/apache/tajo/pull/422/files#diff-23481c6f9849708ac4a57d2252714846R312

        There is a priority about resolving names. If a record column and a table, a table has a higher priority. To resolve this problem, users should use full qualified names.

        Could you explain what alias for nested record is? Is it different from just column alias?

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-83844491 Hi @jihoonson, > It looks that a record column and a table have the same name. In this case, how can we identify them exactly? Here is how to resolve names. https://github.com/apache/tajo/pull/422/files#diff-23481c6f9849708ac4a57d2252714846R312 There is a priority about resolving names. If a record column and a table, a table has a higher priority. To resolve this problem, users should use full qualified names. Could you explain what alias for nested record is? Is it different from just column alias?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-83850557

        @hyunsik thanks. It looks good.
        On alias, I was missed it. My suggestion is exactly same with column alias.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-83850557 @hyunsik thanks. It looks good. On alias, I was missed it. My suggestion is exactly same with column alias.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12705774/TAJO-1359_2.patch
        against master revision release-0.9.0-rc0-208-g725448c.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 20 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 85 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs:
        org.apache.tajo.engine.query.TestSelectNestedRecord

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/624//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-cli.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/624//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12705774/TAJO-1359_2.patch against master revision release-0.9.0-rc0-208-g725448c. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 85 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs: org.apache.tajo.engine.query.TestSelectNestedRecord Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/624//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-cli.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/624//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/624//console This message is automatically generated.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I merged the recent patches for findbug warning fixes.

        Show
        hyunsik Hyunsik Choi added a comment - I merged the recent patches for findbug warning fixes.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12706101/TAJO-1359_3.patch
        against master revision release-0.9.0-rc0-212-g154f5b9.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 20 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 25 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs:
        org.apache.tajo.engine.query.TestSelectNestedRecord

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/629//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/629//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12706101/TAJO-1359_3.patch against master revision release-0.9.0-rc0-212-g154f5b9. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 25 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs: org.apache.tajo.engine.query.TestSelectNestedRecord Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/629//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-plan.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/629//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-client.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/629//console This message is automatically generated.
        Hide
        hyunsik Hyunsik Choi added a comment -

        CI shows findbug warnings. I investigated them all. I made only one warning.

        But, this patch fixes three findbugs warnings, include mine and other trivial things.

        We also need to fix other warning in another jira.

        Show
        hyunsik Hyunsik Choi added a comment - CI shows findbug warnings. I investigated them all. I made only one warning. But, this patch fixes three findbugs warnings, include mine and other trivial things. We also need to fix other warning in another jira.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12706114/TAJO-1359_4.patch
        against master revision release-0.9.0-rc0-212-g154f5b9.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 20 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 22 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs:
        org.apache.tajo.engine.query.TestSelectNestedRecord

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/630//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/630//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12706114/TAJO-1359_4.patch against master revision release-0.9.0-rc0-212-g154f5b9. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 20 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 22 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hcatalog tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs: org.apache.tajo.engine.query.TestSelectNestedRecord Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/630//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-core.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-jdbc.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/630//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-common.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/630//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-85504584

        @hyunsik truly sorry for delayed review.
        I'm reviewing, but it requires to be rebased.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-85504584 @hyunsik truly sorry for delayed review. I'm reviewing, but it requires to be rebased.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I rebased the patch against the recent changes.

        Show
        hyunsik Hyunsik Choi added a comment - I rebased the patch against the recent changes.
        Hide
        hyunsik Hyunsik Choi added a comment -

        The patch is ready to be reviewed even though I'll add more trivial comments.

        Show
        hyunsik Hyunsik Choi added a comment - The patch is ready to be reviewed even though I'll add more trivial comments.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12730780/TAJO-1359_5.patch
        against master revision release-0.9.0-rc0-283-g9b3824b.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 21 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 15 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/772//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/772//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/772//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/772//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/772//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12730780/TAJO-1359_5.patch against master revision release-0.9.0-rc0-283-g9b3824b. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 15 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/772//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/772//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hdfs.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/772//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/772//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/772//console This message is automatically generated.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I fixed some findbug warnings, but I didn't fix all of them found because they are not related to my changes. I think that we should fix them in another jira.

        Show
        hyunsik Hyunsik Choi added a comment - I fixed some findbug warnings, but I didn't fix all of them found because they are not related to my changes. I think that we should fix them in another jira.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12730961/TAJO-1359_6.patch
        against master revision release-0.9.0-rc0-283-g9b3824b.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 21 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs:
        org.apache.tajo.engine.query.TestHBaseTable

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/773//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/773//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/773//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/773//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12730961/TAJO-1359_6.patch against master revision release-0.9.0-rc0-283-g9b3824b. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs: org.apache.tajo.engine.query.TestHBaseTable Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/773//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/773//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/773//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/773//console This message is automatically generated.
        Hide
        hyunsik Hyunsik Choi added a comment -

        I've updated the patch.

        Show
        hyunsik Hyunsik Choi added a comment - I've updated the patch.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12731094/TAJO-1359_7.patch
        against master revision release-0.9.0-rc0-283-g9b3824b.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 21 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/774//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/774//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/774//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/774//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12731094/TAJO-1359_7.patch against master revision release-0.9.0-rc0-283-g9b3824b. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/774//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/774//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/774//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/774//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/422#discussion_r29857765

        — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/NestedPathUtil.java —
        @@ -0,0 +1,109 @@
        +/**
        + * Licensed to the Apache Software Foundation (ASF) under one
        + * or more contributor license agreements. See the NOTICE file
        + * distributed with this work for additional information
        + * regarding copyright ownership. The ASF licenses this file
        + * to you under the Apache License, Version 2.0 (the
        + * "License"); you may not use this file except in compliance
        + * with the License. You may obtain a copy of the License at
        + *
        + * http://www.apache.org/licenses/LICENSE-2.0
        + *
        + * Unless required by applicable law or agreed to in writing, software
        + * distributed under the License is distributed on an "AS IS" BASIS,
        + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
        + * See the License for the specific language governing permissions and
        + * limitations under the License.
        + */
        +
        +package org.apache.tajo.catalog;
        +
        +import com.google.common.base.Preconditions;
        +import org.apache.tajo.common.TajoDataTypes.Type;
        +
        +import java.util.ArrayList;
        +import java.util.Collections;
        +import java.util.List;
        +
        +/**
        + * Utility methods for nested field
        + */
        +public class NestedPathUtil {
        + public static final String PATH_DELIMITER = "/";
        +
        + public static final List<String> ROOT_PATH = Collections.unmodifiableList(new ArrayList<String>());
        +
        + public static boolean isPath(String name)

        { + return name.indexOf(PATH_DELIMITER.charAt(0)) >= 0; + }

        +
        + public static String make(String [] parts)

        { + return make(parts, 0); + }

        +
        + public static String make(String [] parts, int startIndex)

        { + return make(parts, startIndex, parts.length); + }

        +
        + /**
        + * Make a nested field path
        + *
        + * @param parts path parts
        + * @param startIndex startIndex
        + * @param depth Depth
        + * @return Path
        + */
        + public static String make(String [] parts, int startIndex, int depth) {
        — End diff –

        It would be nice if the method name is more intuitive.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/422#discussion_r29857765 — Diff: tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/NestedPathUtil.java — @@ -0,0 +1,109 @@ +/** + * Licensed to the Apache Software Foundation (ASF) under one + * or more contributor license agreements. See the NOTICE file + * distributed with this work for additional information + * regarding copyright ownership. The ASF licenses this file + * to you under the Apache License, Version 2.0 (the + * "License"); you may not use this file except in compliance + * with the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.tajo.catalog; + +import com.google.common.base.Preconditions; +import org.apache.tajo.common.TajoDataTypes.Type; + +import java.util.ArrayList; +import java.util.Collections; +import java.util.List; + +/** + * Utility methods for nested field + */ +public class NestedPathUtil { + public static final String PATH_DELIMITER = "/"; + + public static final List<String> ROOT_PATH = Collections.unmodifiableList(new ArrayList<String>()); + + public static boolean isPath(String name) { + return name.indexOf(PATH_DELIMITER.charAt(0)) >= 0; + } + + public static String make(String [] parts) { + return make(parts, 0); + } + + public static String make(String [] parts, int startIndex) { + return make(parts, startIndex, parts.length); + } + + /** + * Make a nested field path + * + * @param parts path parts + * @param startIndex startIndex + * @param depth Depth + * @return Path + */ + public static String make(String [] parts, int startIndex, int depth) { — End diff – It would be nice if the method name is more intuitive.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/422#discussion_r29860469

        — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java —
        @@ -186,6 +195,7 @@ private void initScanner(Schema projected) throws IOException {
        // set system default properties
        PlannerUtil.applySystemDefaultToTableProperties(context.getQueryContext(), meta);

        + // TODO - If fragments is null, we don't need to make any task. We should investigate it later
        — End diff –

        How about logging a debug message for later investigation?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/422#discussion_r29860469 — Diff: tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java — @@ -186,6 +195,7 @@ private void initScanner(Schema projected) throws IOException { // set system default properties PlannerUtil.applySystemDefaultToTableProperties(context.getQueryContext(), meta); + // TODO - If fragments is null, we don't need to make any task. We should investigate it later — End diff – How about logging a debug message for later investigation?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/422#discussion_r29862163

        — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java —
        @@ -51,30 +70,45 @@
        resolverMap.put(NameResolvingMode.LEGACY, new ResolverByLegacy());
        }

        + public static Column resolve(LogicalPlan plan, LogicalPlan.QueryBlock block, ColumnReferenceExpr column,
        + NameResolvingMode mode) throws PlanningException {
        + if (!resolverMap.containsKey(mode))

        { + throw new PlanningException("Unsupported name resolving level: " + mode.name()); + }

        + return resolverMap.get(mode).resolve(plan, block, column);
        + }
        +
        abstract Column resolve(LogicalPlan plan, LogicalPlan.QueryBlock block, ColumnReferenceExpr columnRef)
        throws PlanningException;

        /**

        • * Try to find the database name
          + * Guess a relation from a table name regardless of whether the given name is qualified or not.
          *
        • @param block the current block
        • * @param tableName The table name
        • * @return The found database name
          + * @param tableName The table name which can be either qualified or not.
          + * @return A corresponding relation
        • @throws PlanningException
          */
        • public static String resolveDatabase(LogicalPlan.QueryBlock block, String tableName) throws PlanningException {
        • List<String> found = new ArrayList<String>();
          + public static RelationNode lookupTable(LogicalPlan.QueryBlock block, String tableName) throws PlanningException {
          + List<RelationNode> found = TUtil.newList();
          +
          for (RelationNode relation : block.getRelations()) {
        • // check alias name or table name
        • if (CatalogUtil.extractSimpleName(relation.getCanonicalName()).equals(tableName) ||
          +
          + // if a table name is qualified
          + if (relation.getCanonicalName().equals(tableName) || relation.getTableName().equals(tableName)) { + found.add(relation); + + // if a table name is not qualified + }

          else if (CatalogUtil.extractSimpleName(relation.getCanonicalName()).equals(tableName) ||
          CatalogUtil.extractSimpleName(relation.getTableName()).equals(tableName)) {

        • // obtain the database name
        • found.add(CatalogUtil.extractQualifier(relation.getTableName()));
          + // obtain the a qualified table name
            • End diff –

        This comment looks unnecessary.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/422#discussion_r29862163 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java — @@ -51,30 +70,45 @@ resolverMap.put(NameResolvingMode.LEGACY, new ResolverByLegacy()); } + public static Column resolve(LogicalPlan plan, LogicalPlan.QueryBlock block, ColumnReferenceExpr column, + NameResolvingMode mode) throws PlanningException { + if (!resolverMap.containsKey(mode)) { + throw new PlanningException("Unsupported name resolving level: " + mode.name()); + } + return resolverMap.get(mode).resolve(plan, block, column); + } + abstract Column resolve(LogicalPlan plan, LogicalPlan.QueryBlock block, ColumnReferenceExpr columnRef) throws PlanningException; /** * Try to find the database name + * Guess a relation from a table name regardless of whether the given name is qualified or not. * @param block the current block * @param tableName The table name * @return The found database name + * @param tableName The table name which can be either qualified or not. + * @return A corresponding relation @throws PlanningException */ public static String resolveDatabase(LogicalPlan.QueryBlock block, String tableName) throws PlanningException { List<String> found = new ArrayList<String>(); + public static RelationNode lookupTable(LogicalPlan.QueryBlock block, String tableName) throws PlanningException { + List<RelationNode> found = TUtil.newList(); + for (RelationNode relation : block.getRelations()) { // check alias name or table name if (CatalogUtil.extractSimpleName(relation.getCanonicalName()).equals(tableName) || + + // if a table name is qualified + if (relation.getCanonicalName().equals(tableName) || relation.getTableName().equals(tableName)) { + found.add(relation); + + // if a table name is not qualified + } else if (CatalogUtil.extractSimpleName(relation.getCanonicalName()).equals(tableName) || CatalogUtil.extractSimpleName(relation.getTableName()).equals(tableName)) { // obtain the database name found.add(CatalogUtil.extractQualifier(relation.getTableName())); + // obtain the a qualified table name End diff – This comment looks unnecessary.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/422#discussion_r29862737

        — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java —
        @@ -162,18 +210,22 @@ static Column resolveFromCurrentAndChildNode(LogicalPlan.QueryBlock block, Colum
        }

        /**

        • * It tries to find a full qualified column name from all relations in the current block.
          + * Lookup a column among all relations in the current block from a column name.
          + *
          + * It assumes that <code>columnName</code> is not any qualified name.
          *
        • @param block The current query block
        • * @param columnRef The column reference to be found
          + * @param columnName The column reference to be found
        • @return The found column
          */
        • static Column resolveFromAllRelsInBlock(LogicalPlan.QueryBlock block,
        • ColumnReferenceExpr columnRef) throws VerifyException {
          + static Column lookupColumnFromAllRelsInBlock(LogicalPlan.QueryBlock block,
          + String columnName) throws VerifyException {
          + Preconditions.checkArgument(CatalogUtil.isSimpleIdentifier(columnName));
            • End diff –

        It would be nice if you add an appropriate error message for pre-condition checking failure.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/422#discussion_r29862737 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java — @@ -162,18 +210,22 @@ static Column resolveFromCurrentAndChildNode(LogicalPlan.QueryBlock block, Colum } /** * It tries to find a full qualified column name from all relations in the current block. + * Lookup a column among all relations in the current block from a column name. + * + * It assumes that <code>columnName</code> is not any qualified name. * @param block The current query block * @param columnRef The column reference to be found + * @param columnName The column reference to be found @return The found column */ static Column resolveFromAllRelsInBlock(LogicalPlan.QueryBlock block, ColumnReferenceExpr columnRef) throws VerifyException { + static Column lookupColumnFromAllRelsInBlock(LogicalPlan.QueryBlock block, + String columnName) throws VerifyException { + Preconditions.checkArgument(CatalogUtil.isSimpleIdentifier(columnName)); End diff – It would be nice if you add an appropriate error message for pre-condition checking failure.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/422#discussion_r29862941

        — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java —
        @@ -31,14 +34,30 @@
        import org.apache.tajo.plan.verifier.VerifyException;
        import org.apache.tajo.plan.logical.RelationNode;
        import org.apache.tajo.util.Pair;
        +import org.apache.tajo.util.StringUtils;
        import org.apache.tajo.util.TUtil;

        -import java.util.ArrayList;
        +import java.util.Collection;
        import java.util.List;
        import java.util.Map;
        +import java.util.Set;

        /**

        • * NameResolver utility
          + * Column name resolution utility. A SQL statement can include many kinds of column names,
          + * defined in different ways. Some column name indicates just a column in a relation.
          + * Another column name includes alias table name or alias column name, derived from some expression.
          + *
          + * This utility ensures that each column name is derived from valid and accessible column, and
          + * it also finds the exact data type of the column.
          + *
          + * Terminology:
          + * <ul>
          + * <li>Qualifier: database name, table name, or both included in a column name</li>
          + * <li>Simple name: just column name without any qualifier</li>
            • End diff –

        I have a question. When a column ```C``` has an alias ```A```, is its simple name ```C``` or ```A```?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/422#discussion_r29862941 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java — @@ -31,14 +34,30 @@ import org.apache.tajo.plan.verifier.VerifyException; import org.apache.tajo.plan.logical.RelationNode; import org.apache.tajo.util.Pair; +import org.apache.tajo.util.StringUtils; import org.apache.tajo.util.TUtil; -import java.util.ArrayList; +import java.util.Collection; import java.util.List; import java.util.Map; +import java.util.Set; /** * NameResolver utility + * Column name resolution utility. A SQL statement can include many kinds of column names, + * defined in different ways. Some column name indicates just a column in a relation. + * Another column name includes alias table name or alias column name, derived from some expression. + * + * This utility ensures that each column name is derived from valid and accessible column, and + * it also finds the exact data type of the column. + * + * Terminology: + * <ul> + * <li>Qualifier: database name, table name, or both included in a column name</li> + * <li>Simple name: just column name without any qualifier</li> End diff – I have a question. When a column ```C``` has an alias ```A```, is its simple name ```C``` or ```A```?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/422#discussion_r29865101

        — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java —
        @@ -240,33 +292,94 @@ static Column resolveAliasedName(LogicalPlan.QueryBlock block, ColumnReferenceEx
        }

        /**

        • * It returns a pair of names, which the first value is $ {database}.${table} and the second value
          - * is a simple column name.
          + * Lookup a qualifier and a canonical name of column.
          + *
          + * It returns a pair of names, which the first value is the qualifier ${database}

          .$

          {table}

          and
          + * the second value is column's simple name.
          *

        • @param block The current block
        • @param columnRef The column name
        • @return A pair of normalized qualifier and column name
        • @throws PlanningException
          */
        • static Pair<String, String> normalizeQualifierAndCanonicalName(LogicalPlan.QueryBlock block,
        • ColumnReferenceExpr columnRef)
          + static Pair<String, String> lookupQualifierAndCanonicalName(LogicalPlan.QueryBlock block,
          + ColumnReferenceExpr columnRef)
          throws PlanningException {
        • String qualifier;
        • String canonicalName;
          + Preconditions.checkArgument(columnRef.hasQualifier(), "ColumnReferenceExpr must be qualified.");
          +
          + String [] qualifierParts = columnRef.getQualifier().split("
          .");
        • if (CatalogUtil.isFQTableName(columnRef.getQualifier())) {
        • qualifier = columnRef.getQualifier();
        • canonicalName = columnRef.getCanonicalName();
          + // This method assumes that column name consists of two or more dot chained names.
          + // In this case, there must be three cases as follows:
          + //
          + // - dbname.tbname.column_name.nested_field...
          + // - tbname.column_name.nested_field...
          + // - column.nested_fieldX...
          +
          + Set<RelationNode> guessedRelations = TUtil.newHashSet();
          +
          + // this position indicates the index of column name in qualifierParts;
          + // It must be 0 or more because a qualified column is always passed to lookupQualifierAndCanonicalName().
          + int columnNamePosition = -1;
          +
          + // check for dbname.tbname.column_name.nested_field
          + if (qualifierParts.length >= 2)
          Unknown macro: { + RelationNode rel = lookupTable(block, CatalogUtil.buildFQName(qualifierParts[0], qualifierParts[1])); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 2; + } + }

          +
          + // check for tbname.column_name.nested_field
          + if (qualifierParts.length >= 1)

          Unknown macro: { + RelationNode rel = lookupTable(block, qualifierParts[0]); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 1; + } + }

          +
          + // column.nested_fieldX...
          + if (guessedRelations.size() == 0 && qualifierParts.length == 1) {

            • End diff –

        I have a question for the condition ```guessedRelations.size() == 0```.
        It looks to find every candidate relation, but stop finding if any relations are found for flat fields. I wonder the reason.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/422#discussion_r29865101 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java — @@ -240,33 +292,94 @@ static Column resolveAliasedName(LogicalPlan.QueryBlock block, ColumnReferenceEx } /** * It returns a pair of names, which the first value is $ {database}.${table} and the second value - * is a simple column name. + * Lookup a qualifier and a canonical name of column. + * + * It returns a pair of names, which the first value is the qualifier ${database} .$ {table} and + * the second value is column's simple name. * @param block The current block @param columnRef The column name @return A pair of normalized qualifier and column name @throws PlanningException */ static Pair<String, String> normalizeQualifierAndCanonicalName(LogicalPlan.QueryBlock block, ColumnReferenceExpr columnRef) + static Pair<String, String> lookupQualifierAndCanonicalName(LogicalPlan.QueryBlock block, + ColumnReferenceExpr columnRef) throws PlanningException { String qualifier; String canonicalName; + Preconditions.checkArgument(columnRef.hasQualifier(), "ColumnReferenceExpr must be qualified."); + + String [] qualifierParts = columnRef.getQualifier().split(" ."); if (CatalogUtil.isFQTableName(columnRef.getQualifier())) { qualifier = columnRef.getQualifier(); canonicalName = columnRef.getCanonicalName(); + // This method assumes that column name consists of two or more dot chained names. + // In this case, there must be three cases as follows: + // + // - dbname.tbname.column_name.nested_field... + // - tbname.column_name.nested_field... + // - column.nested_fieldX... + + Set<RelationNode> guessedRelations = TUtil.newHashSet(); + + // this position indicates the index of column name in qualifierParts; + // It must be 0 or more because a qualified column is always passed to lookupQualifierAndCanonicalName(). + int columnNamePosition = -1; + + // check for dbname.tbname.column_name.nested_field + if (qualifierParts.length >= 2) Unknown macro: { + RelationNode rel = lookupTable(block, CatalogUtil.buildFQName(qualifierParts[0], qualifierParts[1])); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 2; + } + } + + // check for tbname.column_name.nested_field + if (qualifierParts.length >= 1) Unknown macro: { + RelationNode rel = lookupTable(block, qualifierParts[0]); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 1; + } + } + + // column.nested_fieldX... + if (guessedRelations.size() == 0 && qualifierParts.length == 1) { End diff – I have a question for the condition ```guessedRelations.size() == 0```. It looks to find every candidate relation, but stop finding if any relations are found for flat fields. I wonder the reason.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/422#discussion_r29865298

        — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java —
        @@ -240,33 +292,94 @@ static Column resolveAliasedName(LogicalPlan.QueryBlock block, ColumnReferenceEx
        }

        /**

        • * It returns a pair of names, which the first value is $ {database}.${table} and the second value
          - * is a simple column name.
          + * Lookup a qualifier and a canonical name of column.
          + *
          + * It returns a pair of names, which the first value is the qualifier ${database}

          .$

          {table}

          and
          + * the second value is column's simple name.
          *

        • @param block The current block
        • @param columnRef The column name
        • @return A pair of normalized qualifier and column name
        • @throws PlanningException
          */
        • static Pair<String, String> normalizeQualifierAndCanonicalName(LogicalPlan.QueryBlock block,
        • ColumnReferenceExpr columnRef)
          + static Pair<String, String> lookupQualifierAndCanonicalName(LogicalPlan.QueryBlock block,
          + ColumnReferenceExpr columnRef)
          throws PlanningException {
        • String qualifier;
        • String canonicalName;
          + Preconditions.checkArgument(columnRef.hasQualifier(), "ColumnReferenceExpr must be qualified.");
          +
          + String [] qualifierParts = columnRef.getQualifier().split("
          .");
        • if (CatalogUtil.isFQTableName(columnRef.getQualifier())) {
        • qualifier = columnRef.getQualifier();
        • canonicalName = columnRef.getCanonicalName();
          + // This method assumes that column name consists of two or more dot chained names.
          + // In this case, there must be three cases as follows:
          + //
          + // - dbname.tbname.column_name.nested_field...
          + // - tbname.column_name.nested_field...
          + // - column.nested_fieldX...
          +
          + Set<RelationNode> guessedRelations = TUtil.newHashSet();
          +
          + // this position indicates the index of column name in qualifierParts;
          + // It must be 0 or more because a qualified column is always passed to lookupQualifierAndCanonicalName().
          + int columnNamePosition = -1;
          +
          + // check for dbname.tbname.column_name.nested_field
          + if (qualifierParts.length >= 2)
          Unknown macro: { + RelationNode rel = lookupTable(block, CatalogUtil.buildFQName(qualifierParts[0], qualifierParts[1])); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 2; + } + }

          +
          + // check for tbname.column_name.nested_field
          + if (qualifierParts.length >= 1)

          Unknown macro: { + RelationNode rel = lookupTable(block, qualifierParts[0]); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 1; + } + }

          +
          + // column.nested_fieldX...
          + if (guessedRelations.size() == 0 && qualifierParts.length == 1) {

            • End diff –

        Oh, I missed the below part that throws an AmbiguousFieldException when there are two ore more candidate relations.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/422#discussion_r29865298 — Diff: tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java — @@ -240,33 +292,94 @@ static Column resolveAliasedName(LogicalPlan.QueryBlock block, ColumnReferenceEx } /** * It returns a pair of names, which the first value is $ {database}.${table} and the second value - * is a simple column name. + * Lookup a qualifier and a canonical name of column. + * + * It returns a pair of names, which the first value is the qualifier ${database} .$ {table} and + * the second value is column's simple name. * @param block The current block @param columnRef The column name @return A pair of normalized qualifier and column name @throws PlanningException */ static Pair<String, String> normalizeQualifierAndCanonicalName(LogicalPlan.QueryBlock block, ColumnReferenceExpr columnRef) + static Pair<String, String> lookupQualifierAndCanonicalName(LogicalPlan.QueryBlock block, + ColumnReferenceExpr columnRef) throws PlanningException { String qualifier; String canonicalName; + Preconditions.checkArgument(columnRef.hasQualifier(), "ColumnReferenceExpr must be qualified."); + + String [] qualifierParts = columnRef.getQualifier().split(" ."); if (CatalogUtil.isFQTableName(columnRef.getQualifier())) { qualifier = columnRef.getQualifier(); canonicalName = columnRef.getCanonicalName(); + // This method assumes that column name consists of two or more dot chained names. + // In this case, there must be three cases as follows: + // + // - dbname.tbname.column_name.nested_field... + // - tbname.column_name.nested_field... + // - column.nested_fieldX... + + Set<RelationNode> guessedRelations = TUtil.newHashSet(); + + // this position indicates the index of column name in qualifierParts; + // It must be 0 or more because a qualified column is always passed to lookupQualifierAndCanonicalName(). + int columnNamePosition = -1; + + // check for dbname.tbname.column_name.nested_field + if (qualifierParts.length >= 2) Unknown macro: { + RelationNode rel = lookupTable(block, CatalogUtil.buildFQName(qualifierParts[0], qualifierParts[1])); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 2; + } + } + + // check for tbname.column_name.nested_field + if (qualifierParts.length >= 1) Unknown macro: { + RelationNode rel = lookupTable(block, qualifierParts[0]); + if (rel != null) { + guessedRelations.add(rel); + columnNamePosition = 1; + } + } + + // column.nested_fieldX... + if (guessedRelations.size() == 0 && qualifierParts.length == 1) { End diff – Oh, I missed the below part that throws an AmbiguousFieldException when there are two ore more candidate relations.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-99936194

        @hyunsik, thanks for your nice work. I've left some trivial comments.
        In addition, I met an error while running an example in the unit test as follows.
        ```
        default> create external table test (title TEXT, name RECORD ( first_name TEXT, last_name TEXT ) ) USING JSON LOCATION 'hdfs://localhost:7020/test/';
        OK
        default> \d
        test
        default> \d test

        table name: default.test
        table path: hdfs://localhost:7020/test
        store type: JSON
        number of rows: unknown
        volume: 257 B
        Options:
        'text.serde'='org.apache.tajo.storage.json.JsonLineSerDe'

        schema:
        title TEXT
        name RECORD (first_name TEXT, last_name TEXT)

        default> select * from test;
        ERROR: java.lang.NullPointerException
        ```
        Here is the log.
        ```
        2015-05-08 01:47:37,239 INFO org.apache.tajo.master.GlobalEngine: Query: select * from test
        2015-05-08 01:47:37,323 ERROR org.apache.tajo.master.GlobalEngine:
        Stack Trace:
        java.lang.NullPointerException
        at org.apache.tajo.catalog.DDLBuilder.buildSchema(DDLBuilder.java:76)
        at org.apache.tajo.catalog.SchemaUtil.toDisplayString(SchemaUtil.java:161)
        at org.apache.tajo.catalog.TypeDesc.toString(TypeDesc.java:66)
        at org.apache.tajo.catalog.Column.toString(Column.java:159)
        at java.lang.String.valueOf(String.java:2847)
        at java.lang.StringBuilder.append(StringBuilder.java:128)
        at org.apache.tajo.catalog.Schema.toString(Schema.java:503)
        at org.apache.tajo.plan.logical.ScanNode.getPlanString(ScanNode.java:246)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitScan(ExplainLogicalPlanVisitor.java:195)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitScan(ExplainLogicalPlanVisitor.java:32)
        at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:108)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitUnaryNode(ExplainLogicalPlanVisitor.java:130)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitProjection(ExplainLogicalPlanVisitor.java:88)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitProjection(ExplainLogicalPlanVisitor.java:32)
        at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:69)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitRoot(ExplainLogicalPlanVisitor.java:81)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitRoot(ExplainLogicalPlanVisitor.java:32)
        at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:60)
        at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.getBlockPlanStrings(ExplainLogicalPlanVisitor.java:74)
        at org.apache.tajo.plan.LogicalPlan.getLogicalPlanAsString(LogicalPlan.java:344)
        at org.apache.tajo.plan.LogicalPlan.getQueryGraphAsString(LogicalPlan.java:334)
        at org.apache.tajo.plan.LogicalPlan.toString(LogicalPlan.java:366)
        at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:275)
        at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:186)
        at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.submitQuery(TajoMasterClientService.java:287)
        ```

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-99936194 @hyunsik, thanks for your nice work. I've left some trivial comments. In addition, I met an error while running an example in the unit test as follows. ``` default> create external table test (title TEXT, name RECORD ( first_name TEXT, last_name TEXT ) ) USING JSON LOCATION 'hdfs://localhost:7020/test/'; OK default> \d test default> \d test table name: default.test table path: hdfs://localhost:7020/test store type: JSON number of rows: unknown volume: 257 B Options: 'text.serde'='org.apache.tajo.storage.json.JsonLineSerDe' schema: title TEXT name RECORD (first_name TEXT, last_name TEXT) default> select * from test; ERROR: java.lang.NullPointerException ``` Here is the log. ``` 2015-05-08 01:47:37,239 INFO org.apache.tajo.master.GlobalEngine: Query: select * from test 2015-05-08 01:47:37,323 ERROR org.apache.tajo.master.GlobalEngine: Stack Trace: java.lang.NullPointerException at org.apache.tajo.catalog.DDLBuilder.buildSchema(DDLBuilder.java:76) at org.apache.tajo.catalog.SchemaUtil.toDisplayString(SchemaUtil.java:161) at org.apache.tajo.catalog.TypeDesc.toString(TypeDesc.java:66) at org.apache.tajo.catalog.Column.toString(Column.java:159) at java.lang.String.valueOf(String.java:2847) at java.lang.StringBuilder.append(StringBuilder.java:128) at org.apache.tajo.catalog.Schema.toString(Schema.java:503) at org.apache.tajo.plan.logical.ScanNode.getPlanString(ScanNode.java:246) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitScan(ExplainLogicalPlanVisitor.java:195) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitScan(ExplainLogicalPlanVisitor.java:32) at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:108) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitUnaryNode(ExplainLogicalPlanVisitor.java:130) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitProjection(ExplainLogicalPlanVisitor.java:88) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitProjection(ExplainLogicalPlanVisitor.java:32) at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:69) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitRoot(ExplainLogicalPlanVisitor.java:81) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.visitRoot(ExplainLogicalPlanVisitor.java:32) at org.apache.tajo.plan.visitor.BasicLogicalPlanVisitor.visit(BasicLogicalPlanVisitor.java:60) at org.apache.tajo.plan.visitor.ExplainLogicalPlanVisitor.getBlockPlanStrings(ExplainLogicalPlanVisitor.java:74) at org.apache.tajo.plan.LogicalPlan.getLogicalPlanAsString(LogicalPlan.java:344) at org.apache.tajo.plan.LogicalPlan.getQueryGraphAsString(LogicalPlan.java:334) at org.apache.tajo.plan.LogicalPlan.toString(LogicalPlan.java:366) at org.apache.tajo.master.GlobalEngine.createLogicalPlan(GlobalEngine.java:275) at org.apache.tajo.master.GlobalEngine.executeQuery(GlobalEngine.java:186) at org.apache.tajo.master.TajoMasterClientService$TajoMasterClientProtocolServiceHandler.submitQuery(TajoMasterClientService.java:287) ```
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-99991148

        The main cause is that my patch does not support direct projection of record type. In other words, the current work only support projecting primitive type fields in nested records. I'll support this feature and flatten operator in another jira issue. Now, I added logical planner to prohibit the direct projection of record type.

        <sup>**[tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java, line 198 \[r6\]](https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AMBbSEZOQS3d0Cy-r6-198)** ([raw file](https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java#L198)):</sup>
        This was investigated at https://issues.apache.org/jira/browse/TAJO-1422. So, I'll remove this TODO comment.

        <sup>**[tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java, line 56 \[r6\]](https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AME6RwC628sqVQV-r6-56)** ([raw file](https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java#L56)):</sup>
        LogicalPreProcessor and LogicalPlanner changes all column names derived from relations are full qualified names. LogicalPlanner temporarily makes some short names during processing. So, the situation you mentioned won't occur in the current planner.

        <sup>**[tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java, line 104 \[r6\]](https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AME6RwC628sqVQV-r6-104)** ([raw file](https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java#L104)):</sup>
        Done.

        <sup>**[tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java, line 223 \[r6\]](https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AME6RwC628sqVQV-r6-223)** ([raw file](https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java#L223)):</sup>
        This is only for developers misuse instead of users.

        Comments from the [review on Reviewable.io](https://reviewable.io:443/reviews/apache/tajo/422)
        <!-- Sent from Reviewable.io -->

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-99991148 The main cause is that my patch does not support direct projection of record type. In other words, the current work only support projecting primitive type fields in nested records. I'll support this feature and flatten operator in another jira issue. Now, I added logical planner to prohibit the direct projection of record type. — <sup>** [tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java, line 198 \[r6\]] ( https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AMBbSEZOQS3d0Cy-r6-198)** ( [raw file] ( https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java#L198)): </sup> This was investigated at https://issues.apache.org/jira/browse/TAJO-1422 . So, I'll remove this TODO comment. — <sup>** [tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java, line 56 \[r6\]] ( https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AME6RwC628sqVQV-r6-56)** ( [raw file] ( https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java#L56)): </sup> LogicalPreProcessor and LogicalPlanner changes all column names derived from relations are full qualified names. LogicalPlanner temporarily makes some short names during processing. So, the situation you mentioned won't occur in the current planner. — <sup>** [tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java, line 104 \[r6\]] ( https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AME6RwC628sqVQV-r6-104)** ( [raw file] ( https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java#L104)): </sup> Done. — <sup>** [tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java, line 223 \[r6\]] ( https://reviewable.io:443/reviews/apache/tajo/422#-Jo8-AME6RwC628sqVQV-r6-223)** ( [raw file] ( https://github.com/apache/tajo/blob/c5263f4e276abe3e42dbf99bf9dd87e142f67539/tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java#L223)): </sup> This is only for developers misuse instead of users. — Comments from the [review on Reviewable.io] ( https://reviewable.io:443/reviews/apache/tajo/422 ) <!-- Sent from Reviewable.io -->
        Hide
        hyunsik Hyunsik Choi added a comment -

        I reflected your comments.

        Show
        hyunsik Hyunsik Choi added a comment - I reflected your comments.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12731251/TAJO-1359_8.patch
        against master revision release-0.9.0-rc0-283-g9b3824b.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 21 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/775//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/775//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/775//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/775//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12731251/TAJO-1359_8.patch against master revision release-0.9.0-rc0-283-g9b3824b. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 21 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/775//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/775//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/775//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/775//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-100076808

        Thanks. I've verified that join and aggregation queries work well. Really nice patch.
        The one thing I'd like to comment is that it needs to improve the error message.
        Here is an example case.
        ```
        default> select title, name.first_name, name.last_name from test t1, test t2 where t1.name.last_name = t2.name.last_name;
        ERROR: column "title" does not exist
        default> select name.first_name, name.last_name from test t1, test t2 where t1.name.last_name = t2.name.last_name;
        ERROR: column name name.first_name is ambiguous
        ```
        It would be great if the ambiguous exception occurs for the first case.

        Comments from the [review on Reviewable.io](https://reviewable.io:443/reviews/apache/tajo/422)
        <!-- Sent from Reviewable.io -->

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-100076808 Thanks. I've verified that join and aggregation queries work well. Really nice patch. The one thing I'd like to comment is that it needs to improve the error message. Here is an example case. ``` default> select title, name.first_name, name.last_name from test t1, test t2 where t1.name.last_name = t2.name.last_name; ERROR: column "title" does not exist default> select name.first_name, name.last_name from test t1, test t2 where t1.name.last_name = t2.name.last_name; ERROR: column name name.first_name is ambiguous ``` It would be great if the ambiguous exception occurs for the first case. — Comments from the [review on Reviewable.io] ( https://reviewable.io:443/reviews/apache/tajo/422 ) <!-- Sent from Reviewable.io -->
        Hide
        hyunsik Hyunsik Choi added a comment -

        I've updated the patch in github PR.

        Show
        hyunsik Hyunsik Choi added a comment - I've updated the patch in github PR.
        Hide
        tajoqa Tajo QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12731698/TAJO-1359_9.patch
        against master revision release-0.9.0-rc0-288-gddd3921.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 23 new or modified test files.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The applied patch does not increase the total number of javadoc warnings.

        +1 checkstyle. The patch generated 0 code style errors.

        -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs.

        Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/780//testReport/
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/780//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html
        Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/780//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html
        Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/780//console

        This message is automatically generated.

        Show
        tajoqa Tajo QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12731698/TAJO-1359_9.patch against master revision release-0.9.0-rc0-288-gddd3921. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 23 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The applied patch does not increase the total number of javadoc warnings. +1 checkstyle. The patch generated 0 code style errors. -1 findbugs. The patch appears to introduce 9 new Findbugs (version 2.0.3) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in tajo-catalog/tajo-catalog-common tajo-catalog/tajo-catalog-drivers/tajo-hive tajo-cli tajo-client tajo-common tajo-core tajo-jdbc tajo-plan tajo-storage/tajo-storage-common tajo-storage/tajo-storage-hbase tajo-storage/tajo-storage-hdfs. Test results: https://builds.apache.org/job/PreCommit-TAJO-Build/780//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/780//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-hbase.html Findbugs warnings: https://builds.apache.org/job/PreCommit-TAJO-Build/780//artifact/incubator-tajo/patchprocess/newPatchFindbugsWarningstajo-storage-common.html Console output: https://builds.apache.org/job/PreCommit-TAJO-Build/780//console This message is automatically generated.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/422#issuecomment-100771418

        +1 thanks for your nice work!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/422#issuecomment-100771418 +1 thanks for your nice work!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user asfgit closed the pull request at:

        https://github.com/apache/tajo/pull/422

        Show
        githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/tajo/pull/422
        Hide
        hyunsik Hyunsik Choi added a comment -

        I committed to the patch to master branch.

        Show
        hyunsik Hyunsik Choi added a comment - I committed to the patch to master branch.
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #700 (See https://builds.apache.org/job/Tajo-master-build/700/)
        TAJO-1359: Add nested field projector and language extension to project nested record. (hyunsik) (hyunsik: rev 0d1bf41f3aff119af5d95270d735101a81d5721c)

        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageManager.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/DelimitedTextFile.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/avro/AvroScanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/EvalExprNode.java
        • tajo-core/src/test/resources/dataset/TestSelectNestedRecord/tweets/sample1.json
        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect2.sql
        • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsJoinKey1.sql
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerializer.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/AbstractHBaseAppender.java
        • tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample2_ddl.sql
        • tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/GreedyHeuristicJoinOrderAlgorithm.java
        • tajo-core/src/main/resources/webapps/admin/catalogview.jsp
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/CSVFile.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java
        • tajo-common/src/main/java/org/apache/tajo/util/TUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/DistinctGroupbyNode.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hive/src/main/java/org/apache/tajo/catalog/store/HiveCatalogStore.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileStorageManager.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/Schema.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/PartitionedTableRewriter.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinEdge.java
        • CHANGES
        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
        • tajo-core/src/test/java/org/apache/tajo/engine/function/TestFunctionLoader.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsJoinKey1.result
        • tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/expr/RowConstantEval.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestJoinBroadcast.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseBinarySerializerDeserializer.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/sequencefile/SequenceFileScanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/ShuffleFileWriteNode.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java
        • tajo-cli/src/main/java/org/apache/tajo/cli/tsql/commands/DescTableCommand.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/expr/WindowFunctionEval.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/SchemaUtil.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerDe.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java
        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestMergeScanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/ProjectionNode.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java
        • tajo-core/src/test/resources/dataset/TestSelectNestedRecord/sample1/table.json
        • tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java
        • tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleCacheScanner.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineSerDe.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect1.result
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/WindowAggNode.java
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/DataChannel.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/DistinctGroupbyThirdAggregationExec.java
        • tajo-core/src/test/java/org/apache/tajo/ws/rs/resources/TestQueryResultResource.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineSerDe.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/function/python/PythonScriptEngine.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/tweets_ddl.sql
        • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
        • tajo-core/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineDeserializer.java
        • tajo-common/src/main/java/org/apache/tajo/storage/Tuple.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseStorageManager.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/GroupbyNode.java
        • tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultSystemScanner.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/ColumnMapping.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java
        • tajo-core/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java
        • tajo-core/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/NestedPathUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/LogicalPlanVerifier.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogUtil.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample1_ddl.sql
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectNestedRecord.java
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.result
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-core/src/test/resources/queries/TestSelectQuery/testSelectWithParentheses2.sql
        • tajo-core/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.sql
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect1.sql
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/RelationNode.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/builder/DistinctGroupbyBuilder.java
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect2.result
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/TruncateTableNode.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #700 (See https://builds.apache.org/job/Tajo-master-build/700/ ) TAJO-1359 : Add nested field projector and language extension to project nested record. (hyunsik) (hyunsik: rev 0d1bf41f3aff119af5d95270d735101a81d5721c) tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageManager.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/DelimitedTextFile.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/avro/AvroScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/EvalExprNode.java tajo-core/src/test/resources/dataset/TestSelectNestedRecord/tweets/sample1.json tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect2.sql tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsJoinKey1.sql tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerializer.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/AbstractHBaseAppender.java tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample2_ddl.sql tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/GreedyHeuristicJoinOrderAlgorithm.java tajo-core/src/main/resources/webapps/admin/catalogview.jsp tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/CSVFile.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java tajo-common/src/main/java/org/apache/tajo/util/TUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/DistinctGroupbyNode.java tajo-catalog/tajo-catalog-drivers/tajo-hive/src/main/java/org/apache/tajo/catalog/store/HiveCatalogStore.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileStorageManager.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/Schema.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/PartitionedTableRewriter.java tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinEdge.java CHANGES tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-core/src/test/java/org/apache/tajo/engine/function/TestFunctionLoader.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsJoinKey1.result tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/RowConstantEval.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestJoinBroadcast.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseBinarySerializerDeserializer.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/sequencefile/SequenceFileScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/ShuffleFileWriteNode.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java tajo-cli/src/main/java/org/apache/tajo/cli/tsql/commands/DescTableCommand.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/WindowFunctionEval.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/SchemaUtil.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerDe.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestMergeScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/ProjectionNode.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java tajo-core/src/test/resources/dataset/TestSelectNestedRecord/sample1/table.json tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleCacheScanner.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineSerDe.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect1.result tajo-plan/src/main/java/org/apache/tajo/plan/logical/WindowAggNode.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/DataChannel.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/DistinctGroupbyThirdAggregationExec.java tajo-core/src/test/java/org/apache/tajo/ws/rs/resources/TestQueryResultResource.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineSerDe.java tajo-plan/src/main/java/org/apache/tajo/plan/function/python/PythonScriptEngine.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/tweets_ddl.sql tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-core/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineDeserializer.java tajo-common/src/main/java/org/apache/tajo/storage/Tuple.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseStorageManager.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/GroupbyNode.java tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultSystemScanner.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/ColumnMapping.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java tajo-core/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java tajo-core/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4 tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/NestedPathUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/verifier/LogicalPlanVerifier.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogUtil.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample1_ddl.sql tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectNestedRecord.java tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.result tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-core/src/test/resources/queries/TestSelectQuery/testSelectWithParentheses2.sql tajo-core/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.sql tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect1.sql tajo-plan/src/main/java/org/apache/tajo/plan/logical/RelationNode.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/builder/DistinctGroupbyBuilder.java tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect2.result tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/TruncateTableNode.java
        Hide
        hudson Hudson added a comment -

        ABORTED: Integrated in Tajo-master-CODEGEN-build #340 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/340/)
        TAJO-1359: Add nested field projector and language extension to project nested record. (hyunsik) (hyunsik: rev 0d1bf41f3aff119af5d95270d735101a81d5721c)

        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinEdge.java
        • tajo-core/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.result
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect2.result
        • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java
        • tajo-core/src/test/resources/dataset/TestSelectNestedRecord/tweets/sample1.json
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/DataChannel.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectNestedRecord.java
        • tajo-core/src/test/java/org/apache/tajo/ws/rs/resources/TestQueryResultResource.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/verifier/LogicalPlanVerifier.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileStorageManager.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/DelimitedTextFile.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/TruncateTableNode.java
        • tajo-core/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/GreedyHeuristicJoinOrderAlgorithm.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/expr/RowConstantEval.java
        • tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleCacheScanner.java
        • tajo-common/src/main/java/org/apache/tajo/storage/Tuple.java
        • tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultSystemScanner.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestJoinBroadcast.java
        • tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleUtil.java
        • tajo-common/src/main/java/org/apache/tajo/util/TUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/ShuffleFileWriteNode.java
        • tajo-cli/src/main/java/org/apache/tajo/cli/tsql/commands/DescTableCommand.java
        • tajo-core/src/test/resources/queries/TestSelectQuery/testSelectWithParentheses2.sql
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/builder/DistinctGroupbyBuilder.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/SchemaUtil.java
        • tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect1.result
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/AbstractHBaseAppender.java
        • tajo-core/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerDe.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/DistinctGroupbyThirdAggregationExec.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseStorageManager.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/ProjectionNode.java
        • CHANGES
        • tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineDeserializer.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/Schema.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/GroupbyNode.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineSerDe.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/sequencefile/SequenceFileScanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/DistinctGroupbyNode.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.sql
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/avro/AvroScanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/RelationNode.java
        • tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java
        • tajo-core/src/test/java/org/apache/tajo/engine/function/TestFunctionLoader.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample2_ddl.sql
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/tweets_ddl.sql
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerializer.java
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseBinarySerializerDeserializer.java
        • tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsJoinKey1.result
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample1_ddl.sql
        • tajo-core/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java
        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestMergeScanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect1.sql
        • tajo-plan/src/main/java/org/apache/tajo/plan/expr/WindowFunctionEval.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageManager.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java
        • tajo-catalog/tajo-catalog-drivers/tajo-hive/src/main/java/org/apache/tajo/catalog/store/HiveCatalogStore.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/EvalExprNode.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsJoinKey1.sql
        • tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/ColumnMapping.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/function/python/PythonScriptEngine.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineSerDe.java
        • tajo-core/src/main/resources/webapps/admin/catalogview.jsp
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogUtil.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/CSVFile.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/logical/WindowAggNode.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java
        • tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/NestedPathUtil.java
        • tajo-core/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java
        • tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java
        • tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect2.sql
        • tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java
        • tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/PartitionedTableRewriter.java
        • tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java
        • tajo-core/src/test/resources/dataset/TestSelectNestedRecord/sample1/table.json
        Show
        hudson Hudson added a comment - ABORTED: Integrated in Tajo-master-CODEGEN-build #340 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/340/ ) TAJO-1359 : Add nested field projector and language extension to project nested record. (hyunsik) (hyunsik: rev 0d1bf41f3aff119af5d95270d735101a81d5721c) tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinEdge.java tajo-core/src/test/java/org/apache/tajo/engine/planner/TestLogicalPlanner.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineDeserializer.java tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.result tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect2.result tajo-plan/src/main/java/org/apache/tajo/plan/verifier/PreLogicalPlanVerifier.java tajo-core/src/test/resources/dataset/TestSelectNestedRecord/tweets/sample1.json tajo-core/src/main/java/org/apache/tajo/engine/planner/global/DataChannel.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestSelectNestedRecord.java tajo-core/src/test/java/org/apache/tajo/ws/rs/resources/TestQueryResultResource.java tajo-plan/src/main/java/org/apache/tajo/plan/verifier/LogicalPlanVerifier.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/Scanner.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/rcfile/RCFile.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileStorageManager.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/DelimitedTextFile.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/TruncateTableNode.java tajo-core/src/test/java/org/apache/tajo/jdbc/TestTajoJdbc.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/parquet/TajoRecordConverter.java tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/GreedyHeuristicJoinOrderAlgorithm.java tajo-plan/src/main/java/org/apache/tajo/plan/expr/RowConstantEval.java tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleCacheScanner.java tajo-common/src/main/java/org/apache/tajo/storage/Tuple.java tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultSystemScanner.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/FileScanner.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestJoinBroadcast.java tajo-core/src/main/java/org/apache/tajo/engine/utils/TupleUtil.java tajo-common/src/main/java/org/apache/tajo/util/TUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/ShuffleFileWriteNode.java tajo-cli/src/main/java/org/apache/tajo/cli/tsql/commands/DescTableCommand.java tajo-core/src/test/resources/queries/TestSelectQuery/testSelectWithParentheses2.sql tajo-core/src/main/java/org/apache/tajo/engine/planner/global/builder/DistinctGroupbyBuilder.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/SchemaUtil.java tajo-jdbc/src/main/java/org/apache/tajo/jdbc/TajoDatabaseMetaData.java tajo-core/src/test/resources/results/TestSelectNestedRecord/testSelect1.result tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/SeqScanExec.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/AbstractHBaseAppender.java tajo-core/src/main/antlr4/org/apache/tajo/engine/parser/SQLParser.g4 tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerDe.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/DistinctGroupbyThirdAggregationExec.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseStorageManager.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/ProjectionNode.java CHANGES tajo-client/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/FilterPushDownRule.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineDeserializer.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseScanner.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/Schema.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/GroupbyNode.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineSerDe.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/sequencefile/SequenceFileScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/DistinctGroupbyNode.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsGroupbyKey1.sql tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/avro/AvroScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/RelationNode.java tajo-core/src/main/java/org/apache/tajo/master/exec/NonForwardQueryResultFileScanner.java tajo-core/src/test/java/org/apache/tajo/engine/function/TestFunctionLoader.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample2_ddl.sql tajo-core/src/test/resources/queries/TestSelectNestedRecord/tweets_ddl.sql tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/json/JsonLineSerializer.java tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/HBaseBinarySerializerDeserializer.java tajo-core/src/test/resources/results/TestSelectNestedRecord/testNestedFieldAsJoinKey1.result tajo-core/src/test/resources/queries/TestSelectNestedRecord/sample1_ddl.sql tajo-core/src/main/java/org/apache/tajo/engine/parser/SQLAnalyzer.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestMergeScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanner.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect1.sql tajo-plan/src/main/java/org/apache/tajo/plan/expr/WindowFunctionEval.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageManager.java tajo-plan/src/main/java/org/apache/tajo/plan/LogicalPlanPreprocessor.java tajo-catalog/tajo-catalog-drivers/tajo-hive/src/main/java/org/apache/tajo/catalog/store/HiveCatalogStore.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/EvalExprNode.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/testNestedFieldAsJoinKey1.sql tajo-storage/tajo-storage-hbase/src/main/java/org/apache/tajo/storage/hbase/ColumnMapping.java tajo-plan/src/main/java/org/apache/tajo/plan/function/python/PythonScriptEngine.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/TextLineSerDe.java tajo-core/src/main/resources/webapps/admin/catalogview.jsp tajo-core/src/main/java/org/apache/tajo/engine/planner/PhysicalPlannerImpl.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/RowStoreUtil.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/CatalogUtil.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/MergeScanner.java tajo-plan/src/main/java/org/apache/tajo/plan/joinorder/JoinGraph.java tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/CSVFile.java tajo-plan/src/main/java/org/apache/tajo/plan/util/PlannerUtil.java tajo-plan/src/main/java/org/apache/tajo/plan/logical/WindowAggNode.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/ResolverByLegacy.java tajo-catalog/tajo-catalog-common/src/main/java/org/apache/tajo/catalog/NestedPathUtil.java tajo-core/src/test/java/org/apache/tajo/engine/query/TestCTASQuery.java tajo-storage/tajo-storage-hdfs/src/test/java/org/apache/tajo/storage/TestStorages.java tajo-core/src/test/resources/queries/TestSelectNestedRecord/testSelect2.sql tajo-common/src/main/java/org/apache/tajo/util/StringUtils.java tajo-core/src/main/java/org/apache/tajo/engine/planner/global/GlobalPlanner.java tajo-storage/tajo-storage-hdfs/src/main/java/org/apache/tajo/storage/text/CSVLineDeserializer.java tajo-plan/src/main/java/org/apache/tajo/plan/rewrite/rules/PartitionedTableRewriter.java tajo-plan/src/main/java/org/apache/tajo/plan/nameresolver/NameResolver.java tajo-core/src/test/resources/dataset/TestSelectNestedRecord/sample1/table.json

          People

          • Assignee:
            hyunsik Hyunsik Choi
            Reporter:
            hyunsik Hyunsik Choi
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development