Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4282

Allow Impala to create columns whose type has more than 4000 characters

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.6.0, Impala 2.7.0, Impala 2.8.0
    • Fix Version/s: Impala 2.9.0
    • Component/s: Frontend
    • Labels:

      Description

      When creating table with complex datatype having more than 4000 characters in hive we received the following error

      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: Add req_V2` (`CD_ID`,`COMMENT`,`COLUMN_NAME`,`TYPE_NAME`,`INTEGER_IDX`) VALUES (?,?,?,?,?)

      As a workaround I increased the width of the TYPE_NAME column in COLUMNS_V2 table of HMS DB which allowed me to create tables in hive.

      But in impala, the table creation fails in the analysis phase
      HiveServer2Error: AnalysisException: Type of column 'xxx' exceeds maximum type length of 4000 characters

      I see that the MAX_TYPE_NAME_LENGTH is configured to 4000 characters which is causing the problem. Could you please remove this hard limit or make it configurable

      https://github.com/cloudera/Impala/blob/cdh5.7.0-release/fe/src/test/java/com/cloudera/impala/analysis/AnalyzeDDLTest.java#L1416-L1418

        Attachments

          Activity

            People

            • Assignee:
              alex.behm Alexander Behm
              Reporter:
              Venkat Sambath Venkat Sambath
            • Votes:
              2 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: