[HUDI-1484] Escape the partition value in HiveSyncTool

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Component/s: writer-core
• Labels: pull-request-available
• Assignee: pengzhiwei
• Reporter: pengzhiwei
• Created: 21/Dec/20 03:49
• Updated: 29/Dec/20 04:02
• Resolved: 29/Dec/20 04:02

Description

Currently Hudi will encode the partition value when URL_ENCODE_PARTITIONING_OPT_KEY set true. However the HiveSyncTool has not decode the partition value when sync partition to hive. And Hive will encode the partition value twice which lead to an exception when query with hive sql or spark sql.

For example the partition "2020/12/20" will encode to "2020%2F12%2F20" by Hudi. When HiveSyncTool sync the "2020%2F12%2F20" to hive, Hive will encode it to "

"2020%252F12%252F20". This will result to query exception for "select xx from tbl where dt = '2020/12/20'

 

Issue Links

• links to: GitHub Pull Request #2363