Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-25576

Configurable datetime formatter for unix_timestamp, from_unixtime

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 3.1.0, 3.0.0, 3.1.1, 3.1.2, 4.0.0-beta-1
    • 4.0.0
    • None

    Description

      HIVE-25403, HIVE-25458 switched the internal implementation of datetime formatter for unix_timestamp and from_unixtime from java.text.SimpleDateFormat to java.time.format.DateTimeFormatter in order fix some bugs and inconsistencies when the aforementioned functions are combined with other UDFs that have already migrated to use the new modern java.time package.

      The two Java formatters present differences in their behavior leading to different query results. The supported patterns, between the two formatters, are also different something that makes existing queries crash at runtime (after upgrade). Adapting to the new behavior of DateTimeFormatter is a challenging and time-consuming task for end users especially due to the widespread use of the afforementioned unixtime functions.

      Although DateTimeFormatter is a clear improvement over SimpleDateFormat some users still want to retain the old behavior for compatibility reasons thus introducing a property is necessary for facilitating migration.

      The goal of this ticket is to introduce a new property namely hive.datetime.formatter to control the formatter used by unix_timestamp and from_unixtime. By default the new DateTimeFormatter is used while the use of SimpleDateFormat is discouraged. Eventually, SimpleDateFormat will cease to exist.

      Attachments

        Issue Links

          Activity

            People

              zabetak Stamatis Zampetakis
              ashish-kumar-sharma Ashish Sharma
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 3h
                  3h