Hive
  1. Hive
  2. HIVE-3884

Better align columns in DESCRIBE table_name output to make more human-readable

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Minor Minor
    • Resolution: Fixed
    • Affects Version/s: 0.9.0
    • Fix Version/s: 0.11.0
    • Component/s: CLI
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Introduces new DESCRIBE PRETTY table_name command and hive.cli.pretty.output.num.cols conf parameter.

      Description

      If a table contains very long comments or very long column names, the output of DESCRIBE table_name is not aligned nicely. The attached screenshot shows the following two problems:

      1. Rows with long column names do not align well with other columns.
      2. Rows with long comments wrap to the next line, and make it hard to read the output. The wrapping behavior depends on the width of the user's terminal width.

      It will be nice to have a DESCRIBE PRETTY table_name command that will produce nicely formatted output that avoids the two problems mentioned above. It is better to introduce a new DESCRIBE PRETTY command rather than change the behavior of the existing DESCRIBE or DESCRIBE FORMATTED commands, so that we avoid breaking any scripts that automatically parse the output.

      Since the pretty formatting depends on the current terminal width, we need a new hive conf parameter to tell the CLI to auto-detect the current terminal width or to use a fixed width (needed for unit tests).

      1. describe_test_table.png
        46 kB
        Dilip Joseph
      2. HIVE-3884.1.patch.txt
        32 kB
        Dilip Joseph
      3. HIVE-3884.2.patch.txt
        43 kB
        Dilip Joseph

        Activity

        Dilip Joseph created issue -
        Hide
        Dilip Joseph added a comment -

        Output of DESCRIBE test_table for the following table:

        CREATE TABLE test_table(
        col1 INT COMMENT 'col1 one line comment',
        col2 STRING COMMENT 'col2 very long comment that is greater than 80 chars and is likely to spill into multiple lines',
        col3 STRING COMMENT 'col3 very long multi-line comment where each line is very long by itself and is likely to spill
        into multiple lines. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin in dolor nisl, sodales
        adipiscing tortor. Integer venenatis',
        col4 INT COMMENT 'col4 one line comment',
        col5_VeryLongColumnNameThatMessesUpAlignment INT COMMENT 'col5 one line comment',
        col6 STRING COMMENT 'col6 one line comment'
        );

        Show
        Dilip Joseph added a comment - Output of DESCRIBE test_table for the following table: CREATE TABLE test_table( col1 INT COMMENT 'col1 one line comment', col2 STRING COMMENT 'col2 very long comment that is greater than 80 chars and is likely to spill into multiple lines', col3 STRING COMMENT 'col3 very long multi-line comment where each line is very long by itself and is likely to spill into multiple lines. Lorem ipsum dolor sit amet, consectetur adipiscing elit. Proin in dolor nisl, sodales adipiscing tortor. Integer venenatis', col4 INT COMMENT 'col4 one line comment', col5_VeryLongColumnNameThatMessesUpAlignment INT COMMENT 'col5 one line comment', col6 STRING COMMENT 'col6 one line comment' );
        Dilip Joseph made changes -
        Field Original Value New Value
        Attachment describe_test_table.png [ 12564211 ]
        Gang Tim Liu made changes -
        Assignee Gang Tim Liu [ gangtimliu ]
        Dilip Joseph made changes -
        Assignee Gang Tim Liu [ gangtimliu ] Dilip Joseph [ dilipjoseph ]
        Dilip Joseph made changes -
        Attachment HIVE-3884.1.patch.txt [ 12564367 ]
        Hide
        Dilip Joseph added a comment -

        Phabricator diff is at https://reviews.facebook.net/D7851

        Show
        Dilip Joseph added a comment - Phabricator diff is at https://reviews.facebook.net/D7851
        Dilip Joseph made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Release Note Introduces new DESCRIBE PRETTY table_name command and hive.cli.pretty.output.num.cols conf parameter.
        Fix Version/s 0.11.0 [ 12323587 ]
        Hide
        Namit Jain added a comment -

        Comments on phabricator

        Show
        Namit Jain added a comment - Comments on phabricator
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Namit Jain made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Namit Jain added a comment -

        comments

        Show
        Namit Jain added a comment - comments
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Hide
        Dilip Joseph added a comment -

        Added comment about tabs at end of comment output. Minor refactoring.

        Show
        Dilip Joseph added a comment - Added comment about tabs at end of comment output. Minor refactoring.
        Dilip Joseph made changes -
        Attachment HIVE-3884.2.patch.txt [ 12565571 ]
        Dilip Joseph made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Hide
        Namit Jain added a comment - - edited

        Committed. Thanks Dilip

        Show
        Namit Jain added a comment - - edited Committed. Thanks Dilip
        Namit Jain made changes -
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Hadoop Flags Reviewed [ 10343 ]
        Resolution Fixed [ 1 ]
        Hide
        Hudson added a comment -

        Integrated in hive-trunk-hadoop1 #28 (See https://builds.apache.org/job/hive-trunk-hadoop1/28/)
        HIVE-3884 Better align columns in DESCRIBE table_name output to make more
        human-readable (Dilip Joseph via namit) (Revision 1435503)

        Result = ABORTED
        namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1435503
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/conf/hive-default.xml.template
        • /hive/trunk/ql/ivy.xml
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataPrettyFormatUtils.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java
        • /hive/trunk/ql/src/test/queries/clientpositive/describe_pretty.q
        • /hive/trunk/ql/src/test/results/clientpositive/describe_pretty.q.out
        Show
        Hudson added a comment - Integrated in hive-trunk-hadoop1 #28 (See https://builds.apache.org/job/hive-trunk-hadoop1/28/ ) HIVE-3884 Better align columns in DESCRIBE table_name output to make more human-readable (Dilip Joseph via namit) (Revision 1435503) Result = ABORTED namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1435503 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/ql/ivy.xml /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataPrettyFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java /hive/trunk/ql/src/test/queries/clientpositive/describe_pretty.q /hive/trunk/ql/src/test/results/clientpositive/describe_pretty.q.out
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-hadoop2 #75 (See https://builds.apache.org/job/Hive-trunk-hadoop2/75/)
        HIVE-3884 Better align columns in DESCRIBE table_name output to make more
        human-readable (Dilip Joseph via namit) (Revision 1435503)

        Result = FAILURE
        namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1435503
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/conf/hive-default.xml.template
        • /hive/trunk/ql/ivy.xml
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataPrettyFormatUtils.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java
        • /hive/trunk/ql/src/test/queries/clientpositive/describe_pretty.q
        • /hive/trunk/ql/src/test/results/clientpositive/describe_pretty.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-hadoop2 #75 (See https://builds.apache.org/job/Hive-trunk-hadoop2/75/ ) HIVE-3884 Better align columns in DESCRIBE table_name output to make more human-readable (Dilip Joseph via namit) (Revision 1435503) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1435503 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/ql/ivy.xml /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataPrettyFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java /hive/trunk/ql/src/test/queries/clientpositive/describe_pretty.q /hive/trunk/ql/src/test/results/clientpositive/describe_pretty.q.out
        Hide
        Hudson added a comment -

        Integrated in Hive-trunk-h0.21 #1924 (See https://builds.apache.org/job/Hive-trunk-h0.21/1924/)
        HIVE-3884 Better align columns in DESCRIBE table_name output to make more
        human-readable (Dilip Joseph via namit) (Revision 1435503)

        Result = FAILURE
        namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1435503
        Files :

        • /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java
        • /hive/trunk/conf/hive-default.xml.template
        • /hive/trunk/ql/ivy.xml
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataPrettyFormatUtils.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g
        • /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java
        • /hive/trunk/ql/src/test/queries/clientpositive/describe_pretty.q
        • /hive/trunk/ql/src/test/results/clientpositive/describe_pretty.q.out
        Show
        Hudson added a comment - Integrated in Hive-trunk-h0.21 #1924 (See https://builds.apache.org/job/Hive-trunk-h0.21/1924/ ) HIVE-3884 Better align columns in DESCRIBE table_name output to make more human-readable (Dilip Joseph via namit) (Revision 1435503) Result = FAILURE namit : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1435503 Files : /hive/trunk/common/src/java/org/apache/hadoop/hive/conf/HiveConf.java /hive/trunk/conf/hive-default.xml.template /hive/trunk/ql/ivy.xml /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/exec/DDLTask.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/JsonMetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/MetaDataPrettyFormatUtils.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/metadata/formatting/TextMetaDataFormatter.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/DDLSemanticAnalyzer.java /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/parse/Hive.g /hive/trunk/ql/src/java/org/apache/hadoop/hive/ql/plan/DescTableDesc.java /hive/trunk/ql/src/test/queries/clientpositive/describe_pretty.q /hive/trunk/ql/src/test/results/clientpositive/describe_pretty.q.out
        Owen O'Malley made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Transition Time In Source Status Execution Times Last Executer Last Execution Date
        Patch Available Patch Available Open Open
        4d 21h 41m 2 Namit Jain 18/Jan/13 09:25
        Open Open Patch Available Patch Available
        3d 6h 31m 3 Dilip Joseph 18/Jan/13 22:43
        Patch Available Patch Available Resolved Resolved
        9h 26m 1 Namit Jain 19/Jan/13 08:09
        Resolved Resolved Closed Closed
        117d 13h 1 Owen O'Malley 16/May/13 22:09

          People

          • Assignee:
            Dilip Joseph
            Reporter:
            Dilip Joseph
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development