[IMPALA-11113] single_node_perf_run.py throws UnicodeDecodeError for TPCDS dataset

• Type: Bug
• Status: Resolved
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: Impala 4.1.0
• Epic Color: ghx-label-1
• Assignee: Gergely Fürnstáhl
• Reporter: Gergely Fürnstáhl
• Created: 09/Feb/22 09:42
• Updated: 05/Dec/22 09:48
• Resolved: 22/Jul/22 16:05

Description

Possible fix:

https://stackoverflow.com/questions/19872773/unicodedecodeerror-while-using-json-dumps

Exception:

Traceback (most recent call last):
  File "/home/gfurnstahl/Impala/bin/run-workload.py", line 280, in <module>
    json.dump(result_map, f, cls=CustomJSONEncoder)
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/_init_.py", line 189, in dump
    for chunk in iterable:
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/encoder.py", line 332, in _iterencode_list
    for chunk in chunks:
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/encoder.py", line 443, in _iterencode
    for chunk in _iterencode(o, _current_indent_level):
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/encoder.py", line 434, in _iterencode
    for chunk in _iterencode_dict(o, _current_indent_level):
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/encoder.py", line 408, in _iterencode_dict
    for chunk in chunks:
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/json/encoder.py", line 313, in _iterencode_list
    yield buf + _encoder(value)
UnicodeDecodeError: 'utf8' codec can't decode byte 0xc9 in position 47: invalid continuation byte
Traceback (most recent call last):
  File "./bin/single_node_perf_run.py", line 359, in <module>
    main()
  File "./bin/single_node_perf_run.py", line 349, in main
    perf_ab_test(options, args)
  File "./bin/single_node_perf_run.py", line 256, in perf_ab_test
    run_workload(temp_dir, workloads, options)
  File "./bin/single_node_perf_run.py", line 154, in run_workload
    configured_call(run_workload)
  File "./bin/single_node_perf_run.py", line 94, in configured_call
    return subprocess.check_call(["bash", "-c", cmd])
  File "/home/gfurnstahl/Impala/toolchain/toolchain-packages-gcc7.5.0/python-2.7.16/lib/python2.7/subprocess.py", line 190, in check_call
    raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['bash', '-c', 'source /home/gfurnstahl/Impala/bin/impala-config.sh && /home/gfurnstahl/Impala/bin/run-workload.py --workloads=tpcds:10 --impalads=localhost:21000 --results_json_file=/home/gfurnstahl/Impala/perf_results/perf_run_l1WHcn/27a1b4c1203fd1fc7929d23659eed0861703e9e1.json --query_iterations=3 --table_formats=parquet/none --plan_first']' returned non-zero exit status 1