Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-4282

Allow Impala to create columns whose type has more than 4000 characters

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • Impala 2.6.0, Impala 2.7.0, Impala 2.8.0
    • Impala 2.9.0
    • Frontend

    Description

      When creating table with complex datatype having more than 4000 characters in hive we received the following error

      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask. MetaException(message:javax.jdo.JDODataStoreException: Add req_V2` (`CD_ID`,`COMMENT`,`COLUMN_NAME`,`TYPE_NAME`,`INTEGER_IDX`) VALUES (?,?,?,?,?)

      As a workaround I increased the width of the TYPE_NAME column in COLUMNS_V2 table of HMS DB which allowed me to create tables in hive.

      But in impala, the table creation fails in the analysis phase
      HiveServer2Error: AnalysisException: Type of column 'xxx' exceeds maximum type length of 4000 characters

      I see that the MAX_TYPE_NAME_LENGTH is configured to 4000 characters which is causing the problem. Could you please remove this hard limit or make it configurable

      https://github.com/cloudera/Impala/blob/cdh5.7.0-release/fe/src/test/java/com/cloudera/impala/analysis/AnalyzeDDLTest.java#L1416-L1418

      Attachments

        Activity

          People

            alex.behm Alexander Behm
            Venkat Sambath Venkat Sambath
            Votes:
            2 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: