[SPARK-30034] Use Apache Hive 2.3 dependency by default

• Type: Umbrella
• Status: Resolved
• Priority: Blocker
• Resolution: Done
• Affects Version/s: 3.0.0
• Fix Version/s: 3.0.0
• Component/s: Build, SQL
• Labels: correctness release-notes
• Target Version/s: 3.0.0
• Assignee: Dongjoon Hyun
• Reporter: Dongjoon Hyun
• Created: 26/Nov/19 00:52
• Updated: 16/Jun/20 10:45
• Resolved: 10/Jan/20 03:41

Description

Some correctness issue like SPARK-25193 is fixed by `hive-2.3` only.

Issue Links

• blocks: SPARK-20202 Remove references to org.spark-project.hive
• fixes: SPARK-31955 Beeline discard the last line of the sql file when submited to thriftserver via beeline
• supercedes: SPARK-25193 insert overwrite doesn't throw exception when drop old data fails

Sub-Tasks

• Add `hive-1.2` and `hive-2.3` profiles: Resolved
• Update `test-dependencies.sh` to check `hive-1.2/2.3` profile: Resolved
• Publish snapshot/release with `-Phive-2.3` only: Resolved
• Remove `hive-2.3` profile from `sql/hive` module: Resolved
• Support `test-hive1.2` in PR Builder: Resolved
• Update release-script for `hive-1.2/2.3` combination: Resolved
• Adjust Jenkins jobs for `hive-1.2/2.3` combination: Resolved
• Move hive-storage-api dependency from `hive-2.3` to `sql/core`: Resolved
• Update documentation: Resolved
• Update migration guide: Resolved
• Update migration guide and add actionable exception for HIVE-15167: Resolved
• Create migration script for metastore_db: Closed
• insert overwrite doesn't throw exception when drop old data fails: Resolved