[IMPALA-11121] Support loading testdata in JSON format

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Duplicate
• Fix Version/s: Impala 4.3.0
• Component/s: Infrastructure
• Epic Color: ghx-label-1
• Assignee: Zihao Ye
• Reporter: Quanlong Huang
• Created: 14/Feb/22 11:31
• Updated: 07/Sep/23 03:13
• Resolved: 07/Sep/23 03:13

Description

For test coverage, we should generate test data in JSON format and run existing tests on it. We should at least support 'json/none/none'. Other formats like 'json/gzip/none', 'json/snap/none' could be optional.

 The goal of this JIRA is being able to generate sufficient testdata for IMPALA-10798. E.g. we should be able to load JSON tables using

./bin/load-data.py -w functional-query --exploration_strategy=exhaustive --table_formats=json/none/none
./bin/load-data.py -w tpch --exploration_strategy=core --table_formats=json/none/none
./bin/load-data.py -w tpcds --exploration_strategy=core --table_formats=json/none/none

Then in IMPALA-10798, we can trigger tests on the json format, e.g.

impala-py.test --table_formats=json/none tests/query_test/test_scanners.py
impala-py.test --table_formats=json/none tests/query_test/test_scanners_fuzz.py