Spark 2.4 and previous versions use a hybrid calendar - Julian + Gregorian in date/timestamp parsing, functions and expressions. The ticket aims to switch Spark on Proleptic Gregorian calendar, and use java.time classes introduced in Java 8 for timestamp/date manipulations. One of the purpose of switching on Proleptic Gregorian calendar is to conform to SQL standard which supposes such calendar.
Spark 3.0 has switched on Proleptic Gregorian calendar in parsing, formatting, and converting dates and timestamps as well as in extracting sub-components like years, days and etc. It uses Java 8 API classes from the java.time packages that based on ISO chronology . Previous versions of Spark performed those operations by using the hybrid calendar (Julian + Gregorian). The changes might impact on the results for dates and timestamps before October 15, 1582 (Gregorian).