Details

    • Type: Bug
    • Status: Resolved
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: Kudu_Impala
    • Fix Version/s: Impala 2.8.0
    • Component/s: Frontend
    • Labels:
    • Docs Text:
      Doc type differences
    • Target Version:

      Description

      Creating Kudu tables shouldn't allow types not supported by Kudu (e.g. VAR/CHAR, DECIMAL, TIMESTAMP, collection types). The behavior is inconsistent, for some types it throws in the catalog, for VAR/CHAR these become strings. These should all fail w/ analysis exceptions until Kudu supports these types.

      Similarly, external tables cannot contain Kudu types that Impala doesn't support (e.g. UNIXTIME_MICROS, BINARY). This does fail, but we need tests to verify.

        Activity

        Hide
        jrussell John Russell added a comment -

        Does that imply there will be similar limitations & error-handling for CREATE TABLE LIKE and CREATE TABLE LIKE PARQUET when the source tables contain Kudu-disallowed types? Can both of those CREATE TABLE variations accept the clauses and attributes needed to set up a Kudu table?

        Show
        jrussell John Russell added a comment - Does that imply there will be similar limitations & error-handling for CREATE TABLE LIKE and CREATE TABLE LIKE PARQUET when the source tables contain Kudu-disallowed types? Can both of those CREATE TABLE variations accept the clauses and attributes needed to set up a Kudu table?
        Hide
        mjacobs Matthew Jacobs added a comment -

        CT LIKE hasn't been implemented yet. There's a JIRA on it: https://issues.cloudera.org/browse/IMPALA-4052
        I don't think it will make it for 2.8.

        Show
        mjacobs Matthew Jacobs added a comment - CT LIKE hasn't been implemented yet. There's a JIRA on it: https://issues.cloudera.org/browse/IMPALA-4052 I don't think it will make it for 2.8.
        Hide
        mjacobs Matthew Jacobs added a comment -

        commit 32294220c4a0a7a159a70cf8ef313622b7190303
        Author: Matthew Jacobs <mj@cloudera.com>
        Date: Mon Oct 31 16:19:22 2016 -0700

        IMPALA-4379: Fix and test Kudu table type checking, follow up

        The first fix for IMPALA-4379 went in before all comments
        were addressed. First commit: 9b507b6.

        This addresses some follow-up comments about how to handling
        ALTER TABLE setting the storage_handler table property,
        which doesn't really make sense to ever allow.

        Change-Id: I93d04a04483af598b392c28874363e3b0202e1f3
        Reviewed-on: http://gerrit.cloudera.org:8080/4894
        Reviewed-by: Matthew Jacobs <mj@cloudera.com>
        Tested-by: Internal Jenkins

        commit 9b507b6ed6d1a1265544f129dbe75441a2f68783
        Author: Matthew Jacobs <mj@cloudera.com>
        Date: Wed Oct 26 19:08:00 2016 -0700

        IMPALA-4379: Fix and test Kudu table type checking

        Creating Kudu tables shouldn't allow types not supported by
        Kudu (e.g. VARCHAR/CHAR, DECIMAL, TIMESTAMP, collection types).
        The behavior is inconsistent: for some types it throws in
        the catalog, for VARCHAR/CHAR these become strings. This changes
        behavior so that all fail during analysis. Analysis tests
        were added.

        Similarly, external tables cannot contain Kudu types that
        Impala doesn't support (e.g. UNIXTIME_MICROS, BINARY). Tests
        were added to validate this behavior. Note that this
        required upgrading the python Kudu client.

        This also fixes a small corner case with ALTER TABLE:
        ALTER TABLE shouldn't allow Kudu tables to change the
        storage descriptor tblproperty, otherwise the table metadata
        gets in an inconsistent state.

        Tests were added for all of the above.

        Change-Id: I475273cbbf4110db8d0f78ddf9a56abfc6221e3e
        Reviewed-on: http://gerrit.cloudera.org:8080/4857
        Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com>
        Tested-by: Tim Armstrong <tarmstrong@cloudera.com>

        Show
        mjacobs Matthew Jacobs added a comment - commit 32294220c4a0a7a159a70cf8ef313622b7190303 Author: Matthew Jacobs <mj@cloudera.com> Date: Mon Oct 31 16:19:22 2016 -0700 IMPALA-4379 : Fix and test Kudu table type checking, follow up The first fix for IMPALA-4379 went in before all comments were addressed. First commit: 9b507b6. This addresses some follow-up comments about how to handling ALTER TABLE setting the storage_handler table property, which doesn't really make sense to ever allow. Change-Id: I93d04a04483af598b392c28874363e3b0202e1f3 Reviewed-on: http://gerrit.cloudera.org:8080/4894 Reviewed-by: Matthew Jacobs <mj@cloudera.com> Tested-by: Internal Jenkins commit 9b507b6ed6d1a1265544f129dbe75441a2f68783 Author: Matthew Jacobs <mj@cloudera.com> Date: Wed Oct 26 19:08:00 2016 -0700 IMPALA-4379 : Fix and test Kudu table type checking Creating Kudu tables shouldn't allow types not supported by Kudu (e.g. VARCHAR/CHAR, DECIMAL, TIMESTAMP, collection types). The behavior is inconsistent: for some types it throws in the catalog, for VARCHAR/CHAR these become strings. This changes behavior so that all fail during analysis. Analysis tests were added. Similarly, external tables cannot contain Kudu types that Impala doesn't support (e.g. UNIXTIME_MICROS, BINARY). Tests were added to validate this behavior. Note that this required upgrading the python Kudu client. This also fixes a small corner case with ALTER TABLE: ALTER TABLE shouldn't allow Kudu tables to change the storage descriptor tblproperty, otherwise the table metadata gets in an inconsistent state. Tests were added for all of the above. Change-Id: I475273cbbf4110db8d0f78ddf9a56abfc6221e3e Reviewed-on: http://gerrit.cloudera.org:8080/4857 Reviewed-by: Dimitris Tsirogiannis <dtsirogiannis@cloudera.com> Tested-by: Tim Armstrong <tarmstrong@cloudera.com>

          People

          • Assignee:
            mjacobs Matthew Jacobs
            Reporter:
            mjacobs Matthew Jacobs
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development