[SPARK-28611] Histogram's height is diffrent

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Problem
• Affects Version/s: 2.4.3
• Component/s: SQL
• Assignee: Unassigned
• Reporter: Yuming Wang
• Created: 03/Aug/19 14:19
• Updated: 01/Mar/21 07:48
• Resolved: 01/Mar/21 07:48

Description

CREATE TABLE desc_col_table (key int COMMENT 'column_comment') USING PARQUET;
-- Test output for histogram statistics
SET spark.sql.statistics.histogram.enabled=true;
SET spark.sql.statistics.histogram.numBins=2;

INSERT INTO desc_col_table values 1, 2, 3, 4;

ANALYZE TABLE desc_col_table COMPUTE STATISTICS FOR COLUMNS key;

DESC EXTENDED desc_col_table key;

spark-sql> DESC EXTENDED desc_col_table key;
col_name	key
data_type	int
comment	column_comment
min	1
max	4
num_nulls	0
distinct_count	4
avg_col_len	4
max_col_len	4
histogram	height: 4.0, num_of_bins: 2
bin_0	lower_bound: 1.0, upper_bound: 2.0, distinct_count: 2
bin_1	lower_bound: 2.0, upper_bound: 4.0, distinct_count: 2

But our result is:
https://github.com/apache/spark/blob/v2.4.3/sql/core/src/test/resources/sql-tests/results/describe-table-column.sql.out#L231-L242

Issue Links

• relates to: SPARK-28527 Build a Test Framework for Thriftserver