Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-5294

Kudu INSERT partitioning fails with constants

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Impala 2.9.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Frontend
    • Labels:
    • Epic Color:
      ghx-label-9

      Description

      A recent change (IMPALA-3742) introduced a bug where INSERTs into Kudu table will fail with an IllegalStateException if they are inserting a constant value into a partition column and a non-constant value into another partition column.

      The reason is that DistributedPlanner removed the constant exprs from the list of partition exprs before creating the KuduPartitionExpr, but the KuduPartitionExpr expects one expr for each partition column, and so a Precondition check is hit (unless all of the values inserted into partition cols are constant, then we fall back to an unpartitioned insert).

        Issue Links

          Activity

          Hide
          twmarshall Thomas Tauber-Marshall added a comment -

          commit b8c8fb1b439ceffc6e167089184b14559f03699c
          Author: Thomas Tauber-Marshall <tmarshall@cloudera.com>
          Date: Mon May 8 19:10:37 2017 -0700

          IMPALA-5294: Kudu INSERT partitioning fails with constants

          An INSERT into a Kudu table with a constant value being inserted
          into a partition column causes an IllegalStateExcaption. This is
          because DistributedPlanner removes constants from the list of
          partition exprs before creating the KuduPartitionExpr, but
          KuduPartitionExpr expects to get one expr per partition column.

          The fix is to pass the full list of partition exprs into the
          KuduPartitionExpr, instead of the list that has had constants
          removed. This preserves the behavior that if all of the partition
          exprs are constant we fall back to UNPARTITIONED.

          One complication is that if a partition expr is a NullLiteral, it
          must be cast to a specific type to be passed to the BE. The
          InsertStmt will cast the partition exprs to the partition column
          types, but these casts may be lost from the copies of the partition
          exprs stored by the KuduPartitionExpr during reset(). To fix this,
          the KuduPartitionExpr can store the types of the partition cols and
          recast the partition exprs to those types during analyze().

          Change-Id: I12cbb319f9a5c47fdbfee347b47650186b27f8f9
          Reviewed-on: http://gerrit.cloudera.org:8080/6828
          Reviewed-by: Thomas Tauber-Marshall <tmarshall@cloudera.com>
          Tested-by: Impala Public Jenkins

          Show
          twmarshall Thomas Tauber-Marshall added a comment - commit b8c8fb1b439ceffc6e167089184b14559f03699c Author: Thomas Tauber-Marshall <tmarshall@cloudera.com> Date: Mon May 8 19:10:37 2017 -0700 IMPALA-5294 : Kudu INSERT partitioning fails with constants An INSERT into a Kudu table with a constant value being inserted into a partition column causes an IllegalStateExcaption. This is because DistributedPlanner removes constants from the list of partition exprs before creating the KuduPartitionExpr, but KuduPartitionExpr expects to get one expr per partition column. The fix is to pass the full list of partition exprs into the KuduPartitionExpr, instead of the list that has had constants removed. This preserves the behavior that if all of the partition exprs are constant we fall back to UNPARTITIONED. One complication is that if a partition expr is a NullLiteral, it must be cast to a specific type to be passed to the BE. The InsertStmt will cast the partition exprs to the partition column types, but these casts may be lost from the copies of the partition exprs stored by the KuduPartitionExpr during reset(). To fix this, the KuduPartitionExpr can store the types of the partition cols and recast the partition exprs to those types during analyze(). Change-Id: I12cbb319f9a5c47fdbfee347b47650186b27f8f9 Reviewed-on: http://gerrit.cloudera.org:8080/6828 Reviewed-by: Thomas Tauber-Marshall <tmarshall@cloudera.com> Tested-by: Impala Public Jenkins

            People

            • Assignee:
              twmarshall Thomas Tauber-Marshall
              Reporter:
              twmarshall Thomas Tauber-Marshall
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development