Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2042

Impala should check avro.schema.literal/url before allowing modification

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: Impala 2.2
    • Fix Version/s: Impala 2.3.0
    • Component/s: None
    • Labels:

      Description

      Impala allows the TBLPROPERTIES entries avro.schema.literal and avro.schema.url to be modified to any string, which differs to Hive which uses the metastore to check that a valid value has been provided. Impala should implement the same safety check mechanism as Hive.

      Invalid values can cause major problems, such as this example:

      [vm1.jeremybeard.net:21000] > create table test_npe_bug stored as avro tblproperties ('avro.schema.literal' = '{ 
      "name": "test_npe_bug"
      , "type": "record"
      , "fields": [
      {"name": "code", "type": "string"}
      , {"name": "description", "type": "string"}
      , {"name": "total_emp", "type": ["null", "int"]}
      , {"name": "salary", "type": ["null", "int"]}
      ]
      }');
      Query: create table test_npe_bug stored as avro tblproperties ('avro.schema.literal' = '{
      "name": "test_npe_bug"
      , "type": "record"
      , "fields": [
      {"name": "code", "type": "string"}
      , {"name": "description", "type": "string"}
      , {"name": "total_emp", "type": ["null", "int"]}
      , {"name": "salary", "type": ["null", "int"]}
      ]
      }')
      
      Fetched 0 row(s) in 0.08s
      [vm1.jeremybeard.net:21000] > describe test_npe_bug;
      Query: describe test_npe_bug
      +-------------+--------+-------------------+
      | name        | type   | comment           |
      +-------------+--------+-------------------+
      | code        | string | from deserializer |
      | description | string | from deserializer |
      | total_emp   | int    | from deserializer |
      | salary      | int    | from deserializer |
      +-------------+--------+-------------------+
      Fetched 4 row(s) in 3.90s
      [vm1.jeremybeard.net:21000] > alter table test_npe_bug set tblproperties ('avro.schema.literal'='');
      Query: alter table test_npe_bug set tblproperties ('avro.schema.literal'='')
      [vm1.jeremybeard.net:21000] > describe test_npe_bug;
      Query: describe test_npe_bug
      ERROR: AnalysisException: Failed to load metadata for table: default.test_npe_bug
      CAUSED BY: TableLoadingException: Failed to load metadata for table: test_npe_bug
      CAUSED BY: NullPointerException: null
      
      [vm1.jeremybeard.net:21000] > drop table test_npe_bug;
      Query: drop table test_npe_bug
      ERROR: AnalysisException: Failed to load metadata for table: default.test_npe_bug
      CAUSED BY: TableLoadingException: Failed to load metadata for table: test_npe_bug
      CAUSED BY: NullPointerException: null
      

        Attachments

          Activity

            People

            • Assignee:
              HuaisiXu Huaisi Xu
              Reporter:
              jbeard Jacob Evan Beard
            • Votes:
              1 Vote for this issue
              Watchers:
              6 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: