[IMPALA-1695] impala-shell pretty-printing is slow

• Type: Improvement
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Affects Version/s: Impala 2.0
• Component/s: Clients
• Labels: performance shell usability
• Target Version: Product Backlog
• Assignee: Unassigned
• Reporter: Ippokratis Pandis
• Created: 26/Jan/15 07:49
• Updated: 15/Dec/20 20:25

Description

Pretty printing is quite slow. For example, running a query from impala-shell with and w/o -B makes the query run in 14.5s and 2.5s respectively.

Below are part of the profile for the two runs

– run impala-shell (pretty-printing)

    ExecSummary:
Operator       #Hosts   Avg Time   Max Time   #Rows  Est. #Rows  Peak Mem  Est. Peak Mem  Detail
-------------------------------------------------------------------------------------------------------------------------
05:EXCHANGE         1    1.509ms    1.509ms   6.37K          15         0        -1.00 B  UNPARTITIONED
02:AGGREGATE        4  186.561ms  202.751ms   6.37K          15  14.26 MB       10.00 MB  FINALIZE
04:AGGREGATE        4    8.469ms     9.46ms   6.38K          15  14.50 MB       10.00 MB
03:EXCHANGE         4  259.948us  345.133us  11.88K          15         0              0  HASH(score_card_full_parque...
01:AGGREGATE        4  172.837ms  185.338ms  11.88K          15  83.46 MB       10.00 MB
00:SCAN HDFS        4   355.90ms  499.316ms  56.62K          15  91.37 MB      432.00 MB  default.score_card_full_par...
    Query Timeline: 14s823ms
       - Start execution: 108.674us (108.674us)
       - Planning finished: 16.688ms (16.579ms)
       - Submit for admission: 31.348ms (14.660ms)
       - Completed admission: 31.592ms (243.999us)
       - Ready to start remote fragments: 39.506ms (7.914ms)
       - Remote fragments started: 454.784ms (415.277ms)
       - Rows available: 2s031ms (1s576ms)
       - First row fetched: 2s065ms (33.861ms)
       - Unregister query: 14s817ms (12s751ms)
  ImpalaServer:
     - ***ClientFetchWaitTimer: 12s735ms*** <-- high
     - RowMaterializationTimer: 44.388ms

– run impala-shell -B (w/o pretty printing)

    ExecSummary:
Operator       #Hosts   Avg Time   Max Time   #Rows  Est. #Rows  Peak Mem  Est. Peak Mem  Detail
-------------------------------------------------------------------------------------------------------------------------
05:EXCHANGE         1    1.527ms    1.527ms   6.37K          15         0        -1.00 B  UNPARTITIONED
02:AGGREGATE        4   180.55ms  194.285ms   6.37K          15  14.26 MB       10.00 MB  FINALIZE
04:AGGREGATE        4    8.239ms    9.548ms   6.38K          15  14.52 MB       10.00 MB
03:EXCHANGE         4    292.4us  366.995us  11.88K          15         0              0  HASH(score_card_full_parque...
01:AGGREGATE        4   191.19ms  199.422ms  11.88K          15  83.46 MB       10.00 MB
00:SCAN HDFS        4  324.740ms  460.908ms  56.62K          15  91.37 MB      432.00 MB  default.score_card_full_par...
    Query Timeline: 2s440ms
       - Start execution: 91.531us (91.531us)
       - Planning finished: 19.494ms (19.402ms)
       - Submit for admission: 32.238ms (12.743ms)
       - Completed admission: 32.377ms (139.598us)
       - Ready to start remote fragments: 40.827ms (8.449ms)
       - Remote fragments started: 457.854ms (417.27ms)
       - Rows available: 2s038ms (1s581ms)
       - First row fetched: 2s065ms (26.836ms)
       - Unregister query: 2s435ms (369.575ms)
  ImpalaServer:
     - ClientFetchWaitTimer: 342.945ms
     - RowMaterializationTimer: 47.580ms