Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5255

Support representing multilevel partitioning schemes

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • Major
    • Resolution: Unresolved
    • Impala 2.9.0
    • None
    • Frontend
    • ghx-label-2

    Description

      Currently, our DataPartition functionality allows only for a single partition specification (e.g. HASH and the corresponding columns).

      Kudu allows tables to have multilevel partitioning schemes (eg. first HASH then RANGE). In the short term, a pending change (IMPALA-3742) will represent Kudu partitioning schemes with a new TPartitionType of KUDU along with the id of the table to take the partitioning scheme from.

      In the long run, we should allow for representing these multilevel partitioning schemes directly. This will allow us to make smarter choices about when to repartition data, e.g. during a grouping aggregation if the input is from Kudu and already partitioned correctly.

      Attachments

        Issue Links

          relates to

          New Feature - A new feature of the product, which has yet to be developed. IMPALA-3118 Bucketed table support

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          Activity

            People

              Unassigned Unassigned
              twmarshall Thomas Tauber-Marshall
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: