Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-3307

add support for IANA time zone database

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: impala 2.3
    • Fix Version/s: Impala 3.1.0
    • Component/s: Backend
    • Labels:

      Description

      Currently the time zones are hard coded timezone_db.cc and they do not take into account that timezone definitions changed year to year (except for Moscow CDH-19918).

      I suggest moving timezone info into a separate config file, so that admins can update if necessary, plus provide tools for updating it from well-known sources.

      1) Define an impala-friendly file format for timezone data (preferably human-editable as well, even more preferably a format that other similar systems already use)
      2) Create tool to extract timezone data from the IANA tzdata database or /usr/share/zoneinfo
      into the format specified.
      3) File (path, hdfs path) should be part of configuration
      4) backends should load the tzinfo into a quick memory structure (quick lookup by id + date) (maybe load/cache each time zone on demand, most of them will never be used)
      5) all date functions should use this generic tzinfo from memory

      regarding 2), similar tools:
      http://www.oracle.com/technetwork/java/javase/tzupdater-readme-136440.html
      http://dev.mysql.com/doc/refman/5.7/en/mysql-tzinfo-to-sql.html

      regarding 3), some reasons to make this configurable, and making 2) a manual step:

      • tzinfo is not perfectly standardised, automatic solutions might fail on some OSes
      • tzinfo on different hosts might be out of sync. Good luck with debugging such cases...
      • we wouldn't want query results automagically/unexpectedly change on OS upgrade
      • we should give the admins the possibility to override / fine-tune tz data if the applications require doing so.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                attilaj Attila Jeges
                Reporter:
                szama_impala_6295 Marcell Szabo
              • Votes:
                0 Vote for this issue
                Watchers:
                12 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: