[SPARK-24590] Make Jenkins tests passed with hadoop 3 profile

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Duplicate
• Affects Version/s: 2.4.0
• Component/s: Build
• Assignee: Unassigned
• Reporter: Hyukjin Kwon
• Created: 19/Jun/18 02:34
• Updated: 12/Dec/22 17:51
• Resolved: 06/Dec/19 07:23

Description

Currently, some tests are being failed with hadoop-3 profile.

Given PR builder (https://github.com/apache/spark/pull/21441#issuecomment-397818337), it reported:

org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-8020: set sql conf in spark conf
org.apache.spark.sql.hive.HiveSparkSubmitSuite.SPARK-9757 Persist Parquet relation with decimal column
org.apache.spark.sql.hive.HiveSparkSubmitSuite.ConnectionURL
org.apache.spark.sql.hive.StatisticsSuite.SPARK-22745 - read Hive's statistics for partition
org.apache.spark.sql.hive.StatisticsSuite.alter table rename after analyze table
org.apache.spark.sql.hive.StatisticsSuite.alter table SET TBLPROPERTIES after analyze table
org.apache.spark.sql.hive.StatisticsSuite.alter table UNSET TBLPROPERTIES after analyze table
org.apache.spark.sql.hive.client.HiveClientSuites.(It is not a test it is a sbt.testing.SuiteSelector)
org.apache.spark.sql.hive.client.VersionsSuite.success sanity check
org.apache.spark.sql.hive.client.VersionsSuite.hadoop configuration preserved	75 ms
org.apache.spark.sql.hive.client.VersionsSuite.*: * (roughly)
org.apache.spark.sql.hive.execution.HiveCatalogedDDLSuite.basic DDL using locale tr - caseSensitive true
org.apache.spark.sql.hive.execution.HiveDDLSuite.create Hive-serde table and view with unicode columns and comment
org.apache.spark.sql.hive.execution.Hive_2_1_DDLSuite.SPARK-21617: ALTER TABLE for non-compatible DataSource tables
org.apache.spark.sql.hive.execution.Hive_2_1_DDLSuite.SPARK-21617: ALTER TABLE for Hive-compatible DataSource tables
org.apache.spark.sql.hive.execution.Hive_2_1_DDLSuite.SPARK-21617: ALTER TABLE for Hive tables
org.apache.spark.sql.hive.execution.Hive_2_1_DDLSuite.SPARK-21617: ALTER TABLE with incompatible schema on Hive-compatible table
org.apache.spark.sql.hive.execution.Hive_2_1_DDLSuite.(It is not a test it is a sbt.testing.SuiteSelector)
org.apache.spark.sql.hive.execution.SQLQuerySuite.SPARK-18355 Read data from a hive table with a new column - orc
org.apache.spark.sql.hive.execution.SQLQuerySuite.SPARK-18355 Read data from a hive table with a new column - parquet
org.apache.spark.sql.hive.orc.HiveOrcSourceSuite.SPARK-19459/SPARK-18220: read char/varchar column written by Hive

Issue Links

• is superceded by: SPARK-27176 Upgrade hadoop-3's built-in Hive maven dependencies to 2.3.4
• links to: [Github] Pull Request #21588 (HyukjinKwon)
• links to: GitHub Pull Request #21588