[SPARK-3065] Add Locale setting to HiveCompatibilitySuite

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.0.2
• Fix Version/s: 1.1.0
• Component/s: SQL

Environment

CentOS release 6.3 (Final)

• Assignee: Unassigned
• Reporter: Gankun Luo
• Created: 15/Aug/14 07:24
• Updated: 27/Aug/14 22:09
• Resolved: 27/Aug/14 22:09

Description

Run the udf_unix_timestamp of org.apache.spark.sql.hive.execution.HiveCompatibilitySuite testcase
with not "America/Los_Angeles" TimeZone throws：

[info] - udf_unix_timestamp *** FAILED ***
[info] Results do not match for udf_unix_timestamp:
[info] SELECT
[info] '2009 Mar 20 11:30:01 am',
[info] unix_timestamp('2009 Mar 20 11:30:01 am', 'yyyy MMM dd h:mm:ss a')
[info] FROM oneline
[info] == Logical Plan ==
[info] Project 2009 Mar 20 11:30:01 am AS c_0#25,HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFUnixTimeStamp(2009 Mar 20 11:30:01 am,yyyy MMM dd h:mm:ss a) AS c_1#26L
[info] MetastoreRelation default, oneline, None
[info]
[info] == Optimized Logical Plan ==
[info] Project 2009 Mar 20 11:30:01 am AS c_0#25,HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFUnixTimeStamp(2009 Mar 20 11:30:01 am,yyyy MMM dd h:mm:ss a) AS c_1#26L
[info] MetastoreRelation default, oneline, None
[info]
[info] == Physical Plan ==
[info] Project 2009 Mar 20 11:30:01 am AS c_0#25,HiveGenericUdf#org.apache.hadoop.hive.ql.udf.generic.GenericUDFUnixTimeStamp(2009 Mar 20 11:30:01 am,yyyy MMM dd h:mm:ss a) AS c_1#26L
[info] HiveTableScan [], (MetastoreRelation default, oneline, None), None
[info]
[info] Code Generation: false
[info] == RDD ==
[info] (2) MappedRDD[37] at map at HiveContext.scala:350
[info] MapPartitionsRDD[36] at mapPartitions at basicOperators.scala:42
[info] MapPartitionsRDD[35] at mapPartitions at TableReader.scala:112
[info] MappedRDD[34] at map at TableReader.scala:240
[info] HadoopRDD[33] at HadoopRDD at TableReader.scala:230
[info] c_0 c_1
[info] !== HIVE - 1 row(s) == == CATALYST - 1 row(s) ==
[info] !2009 Mar 20 11:30:01 am 1237573801 2009 Mar 20 11:30:01 am NULL (HiveComparisonTest.scala:367)

Issue Links

• links to: [Github] Pull Request #1968 (luogankun)
• links to: [Github] Pull Request #2084 (byF)