[IMPALA-9684] Update query option levels

• Type: Task
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Component/s: Backend
• Target Version: Impala 4.3.0
• Epic Color: ghx-label-12
• Assignee: Unassigned
• Reporter: Tim Armstrong
• Created: 21/Apr/20 17:14
• Updated: 23/Nov/22 12:34

Description

We've accumulated a lot of options and many of them are labelled as more important than they really are.

• The basic level should only include options that change user-facing behaviour. Query tuning, e.g. anything related to runtime filters, should be in advanced.
• "Chicken bits" for stable features should be moved to DEPRECATED. E.g. PARQUET_READ_PAGE_INDEX
• Tuning options that are not commonly used should be moved to deprecated or development (depending on whether we might want to remove it, or whether we might retain it for new features). E.g. PREFETCH_MODE

Query options (defaults shown in []):
	ABORT_ON_ERROR: [0]
	COMPRESSION_CODEC: []
	DEFAULT_FILE_FORMAT: [TEXT]
	DEFAULT_HINTS_INSERT_STATEMENT: []
	DISABLE_CODEGEN: [0]
	DISABLE_HDFS_NUM_ROWS_ESTIMATE: [0]
	DISABLE_ROW_RUNTIME_FILTERING: [0]
	DISABLE_STREAMING_PREAGGREGATIONS: [0]
	DISABLE_UNSAFE_SPILLS: [0]
	EXEC_TIME_LIMIT_S: [0]
	EXPLAIN_LEVEL: [STANDARD]
	IDLE_SESSION_TIMEOUT: [0]
	MAX_ROW_SIZE: [524288]
	MAX_STATEMENT_LENGTH_BYTES: [16777216]
	MEM_LIMIT: [0]
	MT_DOP: []
	NUM_SCANNER_THREADS: [0]
	OPTIMIZE_PARTITION_KEY_SCANS: [0]
	PARQUET_ARRAY_RESOLUTION: [THREE_LEVEL]
	PARQUET_FALLBACK_SCHEMA_RESOLUTION: [POSITION]
	QUERY_TIMEOUT_S: [0]
	REQUEST_POOL: []
	RUNTIME_FILTER_MODE: [GLOBAL]
	RUNTIME_FILTER_WAIT_TIME_MS: [0]
	S3_SKIP_INSERT_STAGING: [1]
	SCRATCH_LIMIT: [-1]
	STATEMENT_EXPRESSION_LIMIT: [250000]
	SYNC_DDL: [0]
	THREAD_RESERVATION_AGGREGATE_LIMIT: [0]
	THREAD_RESERVATION_LIMIT: [3000]
	TIMEZONE: [America/Los_Angeles]

Advanced Query Options:
	APPX_COUNT_DISTINCT: [0]
	BROADCAST_BYTES_LIMIT: [34359738368]
	BUFFER_POOL_LIMIT: []
	CLIENT_IDENTIFIER: Impala Shell v4.0.0-SNAPSHOT (494418c) built on Mon Apr 20 22:53:57 PDT 2020
	COMPUTE_STATS_MIN_SAMPLE_SIZE: [1073741824]
	DEFAULT_JOIN_DISTRIBUTION_MODE: [BROADCAST]
	DEFAULT_SPILLABLE_BUFFER_SIZE: [2097152]
	DEFAULT_TRANSACTIONAL_TYPE: [NONE]
	DISABLE_CODEGEN_ROWS_THRESHOLD: [50000]
	DISABLE_DATA_CACHE: [0]
	DISABLE_HBASE_NUM_ROWS_ESTIMATE: [0]
	ENABLE_CNF_REWRITES: [0]
	ENABLE_EXPR_REWRITES: [1]
	EXEC_SINGLE_NODE_ROWS_THRESHOLD: [100]
	FETCH_ROWS_TIMEOUT_MS: [10000]
	HBASE_CACHE_BLOCKS: [0]
	HBASE_CACHING: [0]
	KUDU_READ_MODE: [DEFAULT]
	KUDU_SNAPSHOT_READ_TIMESTAMP_MICROS: [0]
	MAX_CNF_EXPRS: [0]
	MAX_ERRORS: [100]
	MAX_MEM_ESTIMATE_FOR_ADMISSION: [0]
	MAX_NUM_RUNTIME_FILTERS: [10]
	MIN_SPILLABLE_BUFFER_SIZE: [65536]
	NUM_REMOTE_EXECUTOR_CANDIDATES: [3]
	NUM_ROWS_PRODUCED_LIMIT: [0]
	PARQUET_ANNOTATE_STRINGS_UTF8: [0]
	PARQUET_DICTIONARY_FILTERING: [1]
	PARQUET_FILE_SIZE: [0]
	PARQUET_OBJECT_STORE_SPLIT_SIZE: [268435456]
	PARQUET_PAGE_ROW_COUNT_LIMIT: []
	PARQUET_READ_PAGE_INDEX: [1]
	PARQUET_READ_STATISTICS: [1]
	PARQUET_WRITE_PAGE_INDEX: [1]
	PREAGG_BYTES_LIMIT: [-1]
	PREFETCH_MODE: [HT_BUCKET]
	REPLICA_PREFERENCE: [CACHE_LOCAL]
	RESOURCE_TRACE_RATIO: [0.000000]
	RUNTIME_BLOOM_FILTER_SIZE: [1048576]
	RUNTIME_FILTER_MAX_SIZE: [16777216]
	RUNTIME_FILTER_MIN_SIZE: [1048576]
	SCAN_BYTES_LIMIT: [0]
	SCHEDULE_RANDOM_REPLICA: [0]
	SHUFFLE_DISTINCT_EXPRS: [1]
	SUPPORT_START_OVER: [false]
	TOPN_BYTES_LIMIT: [536870912]

Shell Options
	WRITE_DELIMITED: False
	VERBOSE: True
	LIVE_SUMMARY: False
	OUTPUT_FILE: None
	DELIMITER: \t
	LIVE_PROGRESS: False

Variables:
	No variables defined.