Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-4764

Parquet file with INT_16, etc. logical types not supported by simple SELECT

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 1.6.0
    • 1.10.0
    • Execution - Data Types
    • None

    Description

      Create a Parquet file with the following schema:

      message int16Data

      { required int32 index; required int32 value (INT_16); }

      Store it as int_16.parquet in the local file system. Query it with:

      SELECT * from `local`.`root`.`int_16.parquet`;

      The result, in the web UI, is this error:

      org.apache.drill.common.exceptions.UserRemoteException: SYSTEM ERROR: UnsupportedOperationException: unsupported type: INT32 INT_16 Fragment 0:0 [Error Id: c63f66b4-e5a9-4a35-9ceb-546b74645dd4 on 172.30.1.28:31010]

      The INT_16 logical (or "original") type simply tells consumers of the file that the data is actually a 16-bit signed int. Presumably, this should tell Drill to use the SmallIntVector (or NullableSmallIntVector) class for storage. Without supporting this annotation, even 16-bit integers must be stored as 32-bits within Drill.

      Attachments

        1. int_16.parquet
          0.3 kB
          Paul Rogers
        2. int_8.parquet
          0.3 kB
          Paul Rogers
        3. uint_8.parquet
          0.3 kB
          Paul Rogers
        4. uint_16.parquet
          0.3 kB
          Paul Rogers
        5. uint_32.parquet
          0.3 kB
          Paul Rogers

        Issue Links

          Activity

            People

              sharnyk Serhii Harnyk
              paul-rogers Paul Rogers
              Rahul Kumar Challapalli Rahul Kumar Challapalli
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: