[IMPALA-11332] impala-shell strips trailing whitespace from csv output

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 4.1.0
• Fix Version/s: Impala 4.2.0, Impala 4.1.1
• Component/s: Clients
• Target Version: Impala 4.2.0
• Epic Color: ghx-label-9
• Assignee: Joe McDonnell
• Reporter: Joe McDonnell
• Created: 01/Jun/22 19:50
• Updated: 23/Aug/22 05:51
• Resolved: 12/Jul/22 22:09

Description

A query like this should have trailing whitespace in the impala-shell output:

 

impala-shell -B -q 'select "TEST NODE: Creating table       "'
Starting Impala Shell with no authentication using Python 3.6.9
Warning: live_progress only applies to interactive shell sessions, and is being skipped for now.
Opened TCP connection to localhost:21050
Connected to localhost:21050
Server version: impalad version 4.1.0-SNAPSHOT DEBUG (build 730b7addb84839f2aebf63af501cc9e626f0d1c8)
Query: select "TEST NODE: Creating table       "
Query submitted at: 2022-06-01 12:37:58 (Coordinator: http://joemcdonnell:25000)
Query progress can be monitored at: http://joemcdonnell:25000/query_plan?query_id=3743cf7e04b44e18:5ee28b3b00000000
TEST NODE: Creating table <----- whitespace should exist here
Fetched 1 row(s) in 0.11s

This is due to this rstrip() call here:

 

https://github.com/apache/impala/blob/master/shell/shell_output.py#L93

 

    for row in rows:
      if sys.version_info.major == 2:
        row = [val.encode('utf-8', 'replace') if isinstance(val, unicode) else val
            for val in row]
      writer.writerow(row)
    rows = temp_buffer.getvalue().rstrip() <--------
    temp_buffer.close()
    return rows

 

 

Issue Links

• relates to: IMPALA-11380 impala-shell's VerticalOutputFormatter may incorrectly strip trailing whitespace