Uploaded image for project: 'Tajo'
  1. Tajo
  2. TAJO-1857

Rename the section of 'File Formats' to 'Data Formats' and fill compression section of the 'Table Management' chapter

    Details

    • Type: Task
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.11.1
    • Component/s: Documentation
    • Labels:
      None

      Description

      'File format' is a legacy name, so we need to change it.

      In addiiton, the compression section still remains empty, even though we already have some descriptions in the compression section. This makes many users suffer from the difficulty of finding documents about compression.

        Activity

        Hide
        hyunsik Hyunsik Choi added a comment -

        I think that this issue can be resolved regardless of creating 0.11.0 RC artifact. So, I rescheduled it to 0.11.1.

        Show
        hyunsik Hyunsik Choi added a comment - I think that this issue can be resolved regardless of creating 0.11.0 RC artifact. So, I rescheduled it to 0.11.1.
        Hide
        githubbot ASF GitHub Bot added a comment -

        GitHub user eminency opened a pull request:

        https://github.com/apache/tajo/pull/870

        TAJO-1857: Rename the section of 'File Formats' to 'Data Formats' and fill compression section of the 'Table Management' chapter

        Hi, @jihoonson.
        I modified docs and wrote a compression doc newly.
        Please check them out.

        You can merge this pull request into a Git repository by running:

        $ git pull https://github.com/eminency/tajo TAJO-1857

        Alternatively you can review and apply these changes as the patch at:

        https://github.com/apache/tajo/pull/870.patch

        To close this pull request, make a commit to your master/trunk branch
        with (at least) the following in the commit message:

        This closes #870


        commit 3ca5b9fdcd56a70aaa32df1cedfe689fc48559a3
        Author: Jongyoung Park <eminency@gmail.com>
        Date: 2015-11-20T08:05:22Z

        use 'data format' instead of 'file format'

        commit 334ac58d59e4100f8cb79f5b14cf4e23a7125d39
        Author: Jongyoung Park <eminency@gmail.com>
        Date: 2015-11-20T08:05:51Z

        compression document is written roughly


        Show
        githubbot ASF GitHub Bot added a comment - GitHub user eminency opened a pull request: https://github.com/apache/tajo/pull/870 TAJO-1857 : Rename the section of 'File Formats' to 'Data Formats' and fill compression section of the 'Table Management' chapter Hi, @jihoonson. I modified docs and wrote a compression doc newly. Please check them out. You can merge this pull request into a Git repository by running: $ git pull https://github.com/eminency/tajo TAJO-1857 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/tajo/pull/870.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #870 commit 3ca5b9fdcd56a70aaa32df1cedfe689fc48559a3 Author: Jongyoung Park <eminency@gmail.com> Date: 2015-11-20T08:05:22Z use 'data format' instead of 'file format' commit 334ac58d59e4100f8cb79f5b14cf4e23a7125d39 Author: Jongyoung Park <eminency@gmail.com> Date: 2015-11-20T08:05:51Z compression document is written roughly
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/870#discussion_r45601276

        — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst —
        @@ -1,5 +1,23 @@
        -*********************************
        +***********
        Compression
        -*********************************
        +***********

        -.. todo::
        \ No newline at end of file
        +Using compression makes data size compact and network traffic low. Most of Tajo data types support data compression feature.
        — End diff –

        You may mean ```data formats``` instead of ```data types```.
        Also, ```network traffic low``` seems to be ambiguous. How about changing the first sentence to ```Using compression can make data size compact, thereby enabling efficient use of network bandwidth and storage.```?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/870#discussion_r45601276 — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst — @@ -1,5 +1,23 @@ -********************************* +*********** Compression -********************************* +*********** -.. todo:: \ No newline at end of file +Using compression makes data size compact and network traffic low. Most of Tajo data types support data compression feature. — End diff – You may mean ```data formats``` instead of ```data types```. Also, ```network traffic low``` seems to be ambiguous. How about changing the first sentence to ```Using compression can make data size compact, thereby enabling efficient use of network bandwidth and storage.```?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/870#discussion_r45602281

        — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst —
        @@ -1,5 +1,23 @@
        -*********************************
        +***********
        Compression
        -*********************************
        +***********

        -.. todo::
        \ No newline at end of file
        +Using compression makes data size compact and network traffic low. Most of Tajo data types support data compression feature.
        +Currently, compression configuration affcts only for stored data format and it is specified when a table is created as table meta information.
        +Compression for intermidate data or others is not supported now.
        +
        +===========================================
        +Compression Properties for each Data Format
        +===========================================
        +
        + .. csv-table:: Compression Properties and Codec Class
        — End diff –

        Looks very clear!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/870#discussion_r45602281 — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst — @@ -1,5 +1,23 @@ -********************************* +*********** Compression -********************************* +*********** -.. todo:: \ No newline at end of file +Using compression makes data size compact and network traffic low. Most of Tajo data types support data compression feature. +Currently, compression configuration affcts only for stored data format and it is specified when a table is created as table meta information. +Compression for intermidate data or others is not supported now. + +=========================================== +Compression Properties for each Data Format +=========================================== + + .. csv-table:: Compression Properties and Codec Class — End diff – Looks very clear!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/870#discussion_r45602368

        — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst —
        @@ -1,5 +1,23 @@
        -*********************************
        +***********
        Compression
        -*********************************
        +***********

        -.. todo::
        \ No newline at end of file
        +Using compression makes data size compact and network traffic low. Most of Tajo data types support data compression feature.
        +Currently, compression configuration affcts only for stored data format and it is specified when a table is created as table meta information.
        — End diff –

        ```affcts``` -> ```affects```.
        In addition, it would be good if you add a link to the ```sql_language/ddl.html#create-table``` page.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on a diff in the pull request: https://github.com/apache/tajo/pull/870#discussion_r45602368 — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst — @@ -1,5 +1,23 @@ -********************************* +*********** Compression -********************************* +*********** -.. todo:: \ No newline at end of file +Using compression makes data size compact and network traffic low. Most of Tajo data types support data compression feature. +Currently, compression configuration affcts only for stored data format and it is specified when a table is created as table meta information. — End diff – ```affcts``` -> ```affects```. In addition, it would be good if you add a link to the ```sql_language/ddl.html#create-table``` page.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-158933396

        @eminency, thanks for your patch. I left some comments.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-158933396 @eminency, thanks for your patch. I left some comments.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user eminency commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-159132318

        @jihoonson ,
        I updated what you mentioned and a bit further.
        Please check it out.

        Thanks.

        Show
        githubbot ASF GitHub Bot added a comment - Github user eminency commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-159132318 @jihoonson , I updated what you mentioned and a bit further. Please check it out. Thanks.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-159767211

        +1 LGTM!

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-159767211 +1 LGTM!
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/870#discussion_r45934726

        — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst —
        @@ -1,5 +1,23 @@
        -*********************************
        +***********
        Compression
        -*********************************
        +***********

        -.. todo::
        \ No newline at end of file
        +Using compression can make data size compact, thereby enabling efficient use of network bandwidth and storage. Most of Tajo data formats support data compression feature.
        +Currently, compression configuration affects only for stored data format and it is specified when a table is created as table meta information(See `Create Table <../sql_language/ddl.html#create-table>`_).
        — End diff –

        I suggest ``it is enabled when a table is created with the proper table property.``.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/870#discussion_r45934726 — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst — @@ -1,5 +1,23 @@ -********************************* +*********** Compression -********************************* +*********** -.. todo:: \ No newline at end of file +Using compression can make data size compact, thereby enabling efficient use of network bandwidth and storage. Most of Tajo data formats support data compression feature. +Currently, compression configuration affects only for stored data format and it is specified when a table is created as table meta information(See `Create Table <../sql_language/ddl.html#create-table>`_). — End diff – I suggest ``it is enabled when a table is created with the proper table property.``.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on a diff in the pull request:

        https://github.com/apache/tajo/pull/870#discussion_r45934744

        — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst —
        @@ -1,5 +1,23 @@
        -*********************************
        +***********
        Compression
        -*********************************
        +***********

        -.. todo::
        \ No newline at end of file
        +Using compression can make data size compact, thereby enabling efficient use of network bandwidth and storage. Most of Tajo data formats support data compression feature.
        +Currently, compression configuration affects only for stored data format and it is specified when a table is created as table meta information(See `Create Table <../sql_language/ddl.html#create-table>`_).
        +Compression for intermidate data or others is not supported now.
        — End diff –

        I think it is not necessary here because this section addresses tables.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on a diff in the pull request: https://github.com/apache/tajo/pull/870#discussion_r45934744 — Diff: tajo-docs/src/main/sphinx/table_management/compression.rst — @@ -1,5 +1,23 @@ -********************************* +*********** Compression -********************************* +*********** -.. todo:: \ No newline at end of file +Using compression can make data size compact, thereby enabling efficient use of network bandwidth and storage. Most of Tajo data formats support data compression feature. +Currently, compression configuration affects only for stored data format and it is specified when a table is created as table meta information(See `Create Table <../sql_language/ddl.html#create-table>`_). +Compression for intermidate data or others is not supported now. — End diff – I think it is not necessary here because this section addresses tables.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user hyunsik commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-159767610

        I leaved some comments.

        Show
        githubbot ASF GitHub Bot added a comment - Github user hyunsik commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-159767610 I leaved some comments.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-159774134

        @hyunsik, I'm sorry, but have already committed this patch.
        @eminency, would you open a new Jira issue to address his comment? When you make a PR for the new issue, I'll review ASAP.

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-159774134 @hyunsik, I'm sorry, but have already committed this patch. @eminency, would you open a new Jira issue to address his comment? When you make a PR for the new issue, I'll review ASAP.
        Hide
        jihoonson Jihoon Son added a comment -

        Committed to master and 0.11.1

        Show
        jihoonson Jihoon Son added a comment - Committed to master and 0.11.1
        Hide
        hudson Hudson added a comment -

        FAILURE: Integrated in Tajo-master-CODEGEN-build #602 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/602/)
        TAJO-1857: Rename the section of 'File Formats' to 'Data Formats' and (jihoonson: rev 149a44d85c5bcd7677dfd5db3a3d39644a4c8194)

        • tajo-docs/src/main/sphinx/table_management.rst
        • CHANGES
        • tajo-docs/src/main/sphinx/table_management/file_formats.rst
        • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
        • tajo-docs/src/main/sphinx/table_management/tablespaces.rst
        • tajo-docs/src/main/sphinx/table_management/table_overview.rst
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java
        • tajo-docs/src/main/sphinx/table_management/data_formats.rst
        • tajo-docs/src/main/sphinx/table_management/compression.rst
        Show
        hudson Hudson added a comment - FAILURE: Integrated in Tajo-master-CODEGEN-build #602 (See https://builds.apache.org/job/Tajo-master-CODEGEN-build/602/ ) TAJO-1857 : Rename the section of 'File Formats' to 'Data Formats' and (jihoonson: rev 149a44d85c5bcd7677dfd5db3a3d39644a4c8194) tajo-docs/src/main/sphinx/table_management.rst CHANGES tajo-docs/src/main/sphinx/table_management/file_formats.rst tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-docs/src/main/sphinx/table_management/tablespaces.rst tajo-docs/src/main/sphinx/table_management/table_overview.rst tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java tajo-docs/src/main/sphinx/table_management/data_formats.rst tajo-docs/src/main/sphinx/table_management/compression.rst
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-master-build #989 (See https://builds.apache.org/job/Tajo-master-build/989/)
        TAJO-1857: Rename the section of 'File Formats' to 'Data Formats' and (jihoonson: rev 149a44d85c5bcd7677dfd5db3a3d39644a4c8194)

        • tajo-docs/src/main/sphinx/table_management/file_formats.rst
        • tajo-docs/src/main/sphinx/table_management/compression.rst
        • tajo-docs/src/main/sphinx/table_management/table_overview.rst
        • tajo-docs/src/main/sphinx/table_management/tablespaces.rst
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
        • tajo-docs/src/main/sphinx/table_management.rst
        • tajo-docs/src/main/sphinx/table_management/data_formats.rst
        • CHANGES
        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java
        • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-master-build #989 (See https://builds.apache.org/job/Tajo-master-build/989/ ) TAJO-1857 : Rename the section of 'File Formats' to 'Data Formats' and (jihoonson: rev 149a44d85c5bcd7677dfd5db3a3d39644a4c8194) tajo-docs/src/main/sphinx/table_management/file_formats.rst tajo-docs/src/main/sphinx/table_management/compression.rst tajo-docs/src/main/sphinx/table_management/table_overview.rst tajo-docs/src/main/sphinx/table_management/tablespaces.rst tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java tajo-docs/src/main/sphinx/table_management.rst tajo-docs/src/main/sphinx/table_management/data_formats.rst CHANGES tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
        Hide
        hudson Hudson added a comment -

        SUCCESS: Integrated in Tajo-0.11.1-build #113 (See https://builds.apache.org/job/Tajo-0.11.1-build/113/)
        TAJO-1857: Rename the section of 'File Formats' to 'Data Formats' and (jihoonson: rev c91bfdabc79d719c5f52b83b311488ddcf468f29)

        • tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java
        • CHANGES
        • tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java
        • tajo-docs/src/main/sphinx/table_management.rst
        • tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java
        • tajo-docs/src/main/sphinx/table_management/data_formats.rst
        • tajo-docs/src/main/sphinx/table_management/table_overview.rst
        • tajo-docs/src/main/sphinx/table_management/tablespaces.rst
        • tajo-docs/src/main/sphinx/table_management/file_formats.rst
        • tajo-docs/src/main/sphinx/table_management/compression.rst
        Show
        hudson Hudson added a comment - SUCCESS: Integrated in Tajo-0.11.1-build #113 (See https://builds.apache.org/job/Tajo-0.11.1-build/113/ ) TAJO-1857 : Rename the section of 'File Formats' to 'Data Formats' and (jihoonson: rev c91bfdabc79d719c5f52b83b311488ddcf468f29) tajo-storage/tajo-storage-common/src/main/java/org/apache/tajo/storage/StorageProperty.java CHANGES tajo-core/src/main/java/org/apache/tajo/engine/planner/physical/PhysicalPlanUtil.java tajo-docs/src/main/sphinx/table_management.rst tajo-common/src/main/java/org/apache/tajo/conf/TajoConf.java tajo-docs/src/main/sphinx/table_management/data_formats.rst tajo-docs/src/main/sphinx/table_management/table_overview.rst tajo-docs/src/main/sphinx/table_management/tablespaces.rst tajo-docs/src/main/sphinx/table_management/file_formats.rst tajo-docs/src/main/sphinx/table_management/compression.rst
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user eminency commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-160034320

        @jihoonson I submitted new PR : https://github.com/apache/tajo/pull/881

        Show
        githubbot ASF GitHub Bot added a comment - Github user eminency commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-160034320 @jihoonson I submitted new PR : https://github.com/apache/tajo/pull/881
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user jihoonson commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-160416423

        Thanks. Would you close this PR?

        Show
        githubbot ASF GitHub Bot added a comment - Github user jihoonson commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-160416423 Thanks. Would you close this PR?
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user eminency commented on the pull request:

        https://github.com/apache/tajo/pull/870#issuecomment-160502247

        Sure, I close this PR.

        Show
        githubbot ASF GitHub Bot added a comment - Github user eminency commented on the pull request: https://github.com/apache/tajo/pull/870#issuecomment-160502247 Sure, I close this PR.
        Hide
        githubbot ASF GitHub Bot added a comment -

        Github user eminency closed the pull request at:

        https://github.com/apache/tajo/pull/870

        Show
        githubbot ASF GitHub Bot added a comment - Github user eminency closed the pull request at: https://github.com/apache/tajo/pull/870

          People

          • Assignee:
            eminency Jongyoung Park
            Reporter:
            jihoonson Jihoon Son
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development