[SPARK-42516] Non-captured session time zone in view creation

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 3.3.3, 3.4.0
• Fix Version/s: 3.3.3, 3.4.0
• Component/s: SQL
• Assignee: Max Gekk
• Reporter: Max Gekk
• Created: 21/Feb/23 17:05
• Updated: 22/Feb/23 11:04
• Resolved: 22/Feb/23 11:04

Description

The session time zone config is captured only when it is set explicitly but if it is not the view is instantiated with the current settings. That's might confuse users since query results depends on explicit SQL config settings while creating a view.

The example below portraits the issue:

val viewName = "v1_capture_test"
withView(viewName) {
  assert(get.sessionLocalTimeZone === "America/Los_Angeles")
  createView(viewName,
    """select hour(ts) as H from (
      |  select cast('2022-01-01T00:00:00.000 America/Los_Angeles' as timestamp) as ts
      |)""".stripMargin, Seq("H"))
  withDefaultTimeZone(java.time.ZoneId.of("UTC-09:00")) {
    withSQLConf(SESSION_LOCAL_TIMEZONE.key -> "UTC-10:00") {
      sql(s"select H from $viewName").show(false)
    }
  }
} 

It is expected to output:

+---+
|H  |
+---+
|0  |
+---+ 

but actual output is:

+---+
|H  |
+---+
|8  |
+---+ 

Issue Links

• links to: [Github] Pull Request #40103 (MaxGekk)