Uploaded image for project: 'Kylin'
  1. Kylin
  2. KYLIN-3186

Add support for partitioning columns that combine date and time (e.g. YYYYMMDDHHMISS)

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: v2.2.0
    • Fix Version/s: v2.5.0
    • Component/s: Others
    • Labels:
      None

      Description

      In a multitude of existing enterprise applications partitioning is done on a single column that fuse date and time into a single value (string, integer or big integer). Typical formats are YYYYMMDDHHMM or  YYYYMMDDHHMMSS (e.g. 201801181621 and 20180118154734).
      Such representation is human readable and provides natural sorting of the date/time values.

      Lack of support for such date/time representation requires some ugly workarounds, like creating views that split date and time into separate columns or data copying into tables with different partitioning scheme, none of which is a particularly good solution.
      More over, using views approach on Hive causes severe performance issues, due to inability of Hive optimizer correctly analyze filtering conditions auto-generated by Kylin during the flat table build step.

        Attachments

          Activity

            People

            • Assignee:
              Wayne0101 Chao Long
              Reporter:
              seva_ostapenko Vsevolod Ostapenko
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: