Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2057

Improve error message for incorrect avro decimal column declaration

    Details

      Description

      The example in the current avro decimal documentation doesn't make it clear that the "type" needs to be a complex type. It's easy to make the mistake of including the logical type information at the wrong level.

      Ex:

       create TABLE table1
      STORED AS AVRO
      TBLPROPERTIES (
      'avro.schema.literal'='{
      "type": "record",
      "name": "my_record",
      "fields": [
      {"name": "a", "logicalType": "decimal", "precision": 4, "scale": 2, "type": "bytes"}
      ]}')
      
      ERROR: AnalysisException: Error parsing Avro schema for table 'default.table1': Unsupported type 'bytes' of column 'a'
      

      When that happens the error message saying "Unsupported type 'bytes'..." is confusing since "bytes" is in a way a supported type. It would be great to give a more accurate error message. For example, the error message could say the logicalType field was given at the wrong level.

      Btw, here is a correct version of the example

      create TABLE table1
      STORED AS AVRO
      TBLPROPERTIES (
      'avro.schema.literal'='{
      "type": "record",
      "name": "my_record",
      "fields": [
      {"name": "a", "type": {"type": "bytes", "logicalType": "decimal", "precision": 4, "scale": 2}}
      ]}')
      
      Fetched 0 row(s) in 0.05s
      

      Avro docs: https://avro.apache.org/docs/1.7.7/spec.html#Decimal

        Activity

        Show
        anujphadke Anuj Phadke added a comment - https://github.com/apache/incubator-impala/commit/02fc53d5f0a4d39040ca710c00bae4ca62b73fcd
        Hide
        jbapple Jim Apple added a comment -

        This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09.

        2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that:

        https://github.com/apache/incubator-impala/commits/2.8.0

        Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0?

        Thank you!

        Show
        jbapple Jim Apple added a comment - This is a bulk comment on all issues with Fix Version 2.8.0 that were resolved on or after 2016-12-09. 2.8.0 was branched on December 9, with only two changes to master cherry-picked to the 2.8.0 release branch after that: https://github.com/apache/incubator-impala/commits/2.8.0 Issues fixed after December 9 might not be fixed in 2.8.0. If you are the one who marked this issue Resolved, can you check to see if the patch is in 2.8.0 by using the link above? If the patch is not in 2.8.0, can you change the Fix Version to 2.9.0? Thank you!

          People

          • Assignee:
            anujphadke Anuj Phadke
            Reporter:
            caseyc casey
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development