Hive
  1. Hive
  2. HIVE-7390

Make single quote character optional and configurable in BeeLine CSV/TSV output

    Details

    • Type: New Feature New Feature
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.13.1
    • Fix Version/s: 0.14.0
    • Component/s: Clients
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change
    • Release Note:
      Hide
      --outputformat=[table/vertical/csv/tsv/dsv]
      Format mode for result display. Default is table.
      Usage: beeline --outputformat=tsv

      --delimiterForDSV=DELIMITER
      specify the delimiter for delimiter-separated values output format (default: |)
      Usage: beeline --outputformat=dsv --delimiterForDSV=,

      beeline dsv and delimiterForDSV examples are as followings:
      % bin/beeline
      Hive version 0.11.0-SNAPSHOT by Apache
      beeline> !connect jdbc:hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver
      !connect jdbc:hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver
      Connecting to jdbc:hive2://localhost:10000
      Connected to: Hive (version 0.14.0-SNAPSHOT)
      Driver: Hive (version 0.14.0-SNAPSHOT)
      Transaction isolation: TRANSACTION_REPEATABLE_READ
      HiveServer2 Clients – dsv Example
      0: jdbc:hive2://localhost:10000> create table csv_table(id int, name string, info string) row format delimited fields terminated by '\t';
      No rows affected (0.121 seconds)
      0: jdbc:hive2://localhost:10000> load data local inpath '/root/names' overwrite into table csv_table;
      No rows affected (0.245 seconds)
      0: jdbc:hive2://localhost:10000> select * from csv_table;
      +---------------+-----------------+-----------------+--+
      | csv_table.id | csv_table.name | csv_table.info |
      +---------------+-----------------+-----------------+--+
      | 19630001 | "john" | lennon |
      | 19630002 | peter,paul | mccartney |
      | 19630003 | george | harrison |
      | 19630004 | ringo | starr |
      +---------------+-----------------+-----------------+--+
      4 rows selected (0.09 seconds)
      0: jdbc:hive2://localhost:10000> !outformat csv
      Unknown command: outformat csv
      0: jdbc:hive2://localhost:10000> !outputformat csv
      0: jdbc:hive2://localhost:10000> select * from csv_table;
      csv_table.id,csv_table.name,csv_table.info
      19630001,"""john""",lennon
      19630002,"peter,paul",mccartney
      19630003,george,harrison
      19630004,ringo,starr
      4 rows selected (0.105 seconds)
      0: jdbc:hive2://localhost:10000> !outputformat dsv
      0: jdbc:hive2://localhost:10000> select * from csv_table;
      csv_table.id|csv_table.name|csv_table.info
      19630001|"""john"""|lennon
      19630002|peter,paul|mccartney
      19630003|george|harrison
      19630004|ringo|starr
      4 rows selected (0.123 seconds)
      0: jdbc:hive2://localhost:10000> !set delimiterForDSV ',';
      0: jdbc:hive2://localhost:10000> select * from csv_table;
      csv_table.id'csv_table.name'csv_table.info
      19630001'"""john"""'lennon
      19630002'peter,paul'mccartney
      19630003'george'harrison
      19630004'ringo'starr
      4 rows selected (0.11 seconds)
      Show
      --outputformat=[table/vertical/csv/tsv/dsv] Format mode for result display. Default is table. Usage: beeline --outputformat=tsv --delimiterForDSV=DELIMITER specify the delimiter for delimiter-separated values output format (default: |) Usage: beeline --outputformat=dsv --delimiterForDSV=, beeline dsv and delimiterForDSV examples are as followings: % bin/beeline Hive version 0.11.0-SNAPSHOT by Apache beeline> !connect jdbc: hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver !connect jdbc: hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver Connecting to jdbc: hive2://localhost:10000 Connected to: Hive (version 0.14.0-SNAPSHOT) Driver: Hive (version 0.14.0-SNAPSHOT) Transaction isolation: TRANSACTION_REPEATABLE_READ HiveServer2 Clients – dsv Example 0: jdbc: hive2://localhost:10000 > create table csv_table(id int, name string, info string) row format delimited fields terminated by '\t'; No rows affected (0.121 seconds) 0: jdbc: hive2://localhost:10000 > load data local inpath '/root/names' overwrite into table csv_table; No rows affected (0.245 seconds) 0: jdbc: hive2://localhost:10000 > select * from csv_table; +---------------+-----------------+-----------------+--+ | csv_table.id | csv_table.name | csv_table.info | +---------------+-----------------+-----------------+--+ | 19630001 | "john" | lennon | | 19630002 | peter,paul | mccartney | | 19630003 | george | harrison | | 19630004 | ringo | starr | +---------------+-----------------+-----------------+--+ 4 rows selected (0.09 seconds) 0: jdbc: hive2://localhost:10000 > !outformat csv Unknown command: outformat csv 0: jdbc: hive2://localhost:10000 > !outputformat csv 0: jdbc: hive2://localhost:10000 > select * from csv_table; csv_table.id,csv_table.name,csv_table.info 19630001,"""john""",lennon 19630002,"peter,paul",mccartney 19630003,george,harrison 19630004,ringo,starr 4 rows selected (0.105 seconds) 0: jdbc: hive2://localhost:10000 > !outputformat dsv 0: jdbc: hive2://localhost:10000 > select * from csv_table; csv_table.id|csv_table.name|csv_table.info 19630001|"""john"""|lennon 19630002|peter,paul|mccartney 19630003|george|harrison 19630004|ringo|starr 4 rows selected (0.123 seconds) 0: jdbc: hive2://localhost:10000 > !set delimiterForDSV ','; 0: jdbc: hive2://localhost:10000 > select * from csv_table; csv_table.id'csv_table.name'csv_table.info 19630001'"""john"""'lennon 19630002'peter,paul'mccartney 19630003'george'harrison 19630004'ringo'starr 4 rows selected (0.11 seconds)

      Description

      Currently when either the CSV or TSV output formats are used in beeline each column is wrapped in single quotes. Quote wrapping of columns should be optional and the user should be able to choose the character used to wrap the columns.

      1. HIVE-7390.patch
        4 kB
        Jim Halfpenny
      2. HIVE-7390.1.patch
        7 kB
        ferdinand xu
      3. HIVE-7390.2.patch
        7 kB
        ferdinand xu
      4. HIVE-7390.3.patch
        7 kB
        ferdinand xu
      5. HIVE-7390.4.patch
        8 kB
        Ferdinand Xu
      6. HIVE-7390.5.patch
        9 kB
        Ferdinand Xu
      7. HIVE-7390.6.patch
        9 kB
        Ferdinand Xu
      8. HIVE-7390.7.patch
        9 kB
        Ferdinand Xu
      9. HIVE-7390.8.patch
        9 kB
        Ferdinand Xu
      10. HIVE-7390.9.patch
        9 kB
        Ferdinand Xu

        Issue Links

          Activity

          Jim Halfpenny created issue -
          Jim Halfpenny made changes -
          Field Original Value New Value
          Attachment HIVE-7390.patch [ 12655203 ]
          Xuefu Zhang made changes -
          Link This issue is duplicated by HIVE-7434 [ HIVE-7434 ]
          ferdinand xu made changes -
          Attachment HIVE-7390.1.patch [ 12657091 ]
          ferdinand xu made changes -
          Attachment HIVE-7390.2.patch [ 12658322 ]
          ferdinand xu made changes -
          Attachment HIVE-7390.3.patch [ 12658329 ]
          Ferdinand Xu made changes -
          Assignee Ferdinand Xu [ ferd ]
          Ferdinand Xu made changes -
          Attachment HIVE-7390.4.patch [ 12658619 ]
          Ferdinand Xu made changes -
          Attachment HIVE-7390.5.patch [ 12658846 ]
          Ferdinand Xu made changes -
          Attachment HIVE-7390.6.patch [ 12659079 ]
          Ferdinand Xu made changes -
          Attachment HIVE-7390.7.patch [ 12659175 ]
          Ferdinand Xu made changes -
          Status Open [ 1 ] In Progress [ 3 ]
          Ferdinand Xu made changes -
          Status In Progress [ 3 ] Patch Available [ 10002 ]
          Ferdinand Xu made changes -
          Attachment HIVE-7390.8.patch [ 12659597 ]
          Ferdinand Xu made changes -
          Attachment HIVE-7390.9.patch [ 12660905 ]
          Szehon Ho made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Fix Version/s 0.14.0 [ 12326450 ]
          Resolution Fixed [ 1 ]
          Lefty Leverenz made changes -
          Labels TODOC14
          Vaibhav Gumashta made changes -
          Link This issue breaks HIVE-8544 [ HIVE-8544 ]
          Ferdinand Xu made changes -
          Release Note --outputformat=[table/vertical/csv/tsv/dsv]
          Format mode for result display. Default is table.
          Usage: beeline --outputformat=tsv

          --delimiterForDSV=DELIMITER
          specify the delimiter for delimiter-separated values output format (default: |)
          Usage: beeline --outputformat=dsv --delimiterForDSV=,

          beeline dsv and delimiterForDSV examples are as followings:
          % bin/beeline
          Hive version 0.11.0-SNAPSHOT by Apache
          beeline> !connect jdbc:hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver
          !connect jdbc:hive2://localhost:10000 scott tiger org.apache.hive.jdbc.HiveDriver
          Connecting to jdbc:hive2://localhost:10000
          Connected to: Hive (version 0.14.0-SNAPSHOT)
          Driver: Hive (version 0.14.0-SNAPSHOT)
          Transaction isolation: TRANSACTION_REPEATABLE_READ
          HiveServer2 Clients – dsv Example
          0: jdbc:hive2://localhost:10000> create table csv_table(id int, name string, info string) row format delimited fields terminated by '\t';
          No rows affected (0.121 seconds)
          0: jdbc:hive2://localhost:10000> load data local inpath '/root/names' overwrite into table csv_table;
          No rows affected (0.245 seconds)
          0: jdbc:hive2://localhost:10000> select * from csv_table;
          +---------------+-----------------+-----------------+--+
          | csv_table.id | csv_table.name | csv_table.info |
          +---------------+-----------------+-----------------+--+
          | 19630001 | "john" | lennon |
          | 19630002 | peter,paul | mccartney |
          | 19630003 | george | harrison |
          | 19630004 | ringo | starr |
          +---------------+-----------------+-----------------+--+
          4 rows selected (0.09 seconds)
          0: jdbc:hive2://localhost:10000> !outformat csv
          Unknown command: outformat csv
          0: jdbc:hive2://localhost:10000> !outputformat csv
          0: jdbc:hive2://localhost:10000> select * from csv_table;
          csv_table.id,csv_table.name,csv_table.info
          19630001,"""john""",lennon
          19630002,"peter,paul",mccartney
          19630003,george,harrison
          19630004,ringo,starr
          4 rows selected (0.105 seconds)
          0: jdbc:hive2://localhost:10000> !outputformat dsv
          0: jdbc:hive2://localhost:10000> select * from csv_table;
          csv_table.id|csv_table.name|csv_table.info
          19630001|"""john"""|lennon
          19630002|peter,paul|mccartney
          19630003|george|harrison
          19630004|ringo|starr
          4 rows selected (0.123 seconds)
          0: jdbc:hive2://localhost:10000> !set delimiterForDSV ',';
          0: jdbc:hive2://localhost:10000> select * from csv_table;
          csv_table.id'csv_table.name'csv_table.info
          19630001'"""john"""'lennon
          19630002'peter,paul'mccartney
          19630003'george'harrison
          19630004'ringo'starr
          4 rows selected (0.11 seconds)
          Thejas M Nair made changes -
          Hadoop Flags Incompatible change [ 10342 ]
          Thejas M Nair made changes -
          Link This issue breaks HIVE-8615 [ HIVE-8615 ]
          Szehon Ho made changes -
          Labels TODOC14
          Thejas M Nair made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Brock Noland made changes -
          Summary Make quote character optional and configurable in BeeLine CSV/TSV output Make single quote character optional and configurable in BeeLine CSV/TSV output
          Brock Noland made changes -
          Link This issue is related to HIVE-9788 [ HIVE-9788 ]

            People

            • Assignee:
              Ferdinand Xu
              Reporter:
              Jim Halfpenny
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development