Uploaded image for project: 'Apache Drill'
  1. Apache Drill
  2. DRILL-5495

convert_from function on top of int96 data results in ArrayIndexOutOfBoundsException

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.10.0
    • Fix Version/s: 1.14.0
    • Component/s: None
    • Labels:

      Description

      git.commit.id.abbrev=1e0a14c

      The data set used is generated from spark and contains a timestamp stored as int96

      [root@qa-node190 framework]# /home/parquet-tools-1.5.1-SNAPSHOT/parquet-meta /home/framework/framework/resources/Datasources/parquet_date/spark_generated/d4/part-r-00000-08c5c621-62ea-4fee-b690-11576eddc39c.snappy.parquet 
      creator:     parquet-mr (build 32c46643845ea8a705c35d4ec8fc654cc8ff816d) 
      extra:       org.apache.spark.sql.parquet.row.metadata = {"type":"struct","fields":[{"name":"a","type":"integer","nullable":true,"metadata":{}},{"name":"b","type":"strin [more]...
      
      file schema: spark_schema 
      -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      a:           OPTIONAL INT32 R:0 D:1
      b:           OPTIONAL BINARY O:UTF8 R:0 D:1
      c:           OPTIONAL INT32 O:DATE R:0 D:1
      d:           OPTIONAL INT96 R:0 D:1
      
      row group 1: RC:10000 TS:8661 
      -----------------------------------------------------------------------------------------------------------------------------------------------------------------------------------
      a:            INT32 SNAPPY DO:0 FPO:4 SZ:2367/2571/1.09 VC:10000 ENC:RLE,PLAIN,BIT_PACKED
      b:            BINARY SNAPPY DO:0 FPO:2371 SZ:2329/2843/1.22 VC:10000 ENC:RLE,PLAIN_DICTIONARY,BIT_PACKED
      c:            INT32 SNAPPY DO:0 FPO:4700 SZ:1374/1507/1.10 VC:10000 ENC:RLE,PLAIN,BIT_PACKED
      d:            INT96 SNAPPY DO:0 FPO:6074 SZ:1597/1740/1.09 VC:10000 ENC:RLE,PLAIN_DICTIONARY,BIT_PACKED
      

      The below query fails with an ArrayIndexOutOfBoundsException

      select convert_from(d, 'TIMESTAMP_IMPALA') from dfs.`/drill/testdata/resource-manager/d4`;
      
      Fails with below error after displaying a bunch of records
      Error: SYSTEM ERROR: ArrayIndexOutOfBoundsException: 0
      
      Fragment 1:0
      
      [Error Id: f963f6c0-3306-49a6-9d98-a193c5e7cfee on qa-node190.qa.lab:31010] (state=,code=0)
      

      Attached the logs, profiles and data files

        Attachments

        1. drillbit.log
          330 kB
          Rahul Challapalli
        2. d4.tar.gz
          56 kB
          Rahul Challapalli
        3. 26edf56f-6bc6-1e1f-5aa4-d98aec858a4a.sys.drill
          3 kB
          Rahul Challapalli

          Issue Links

            Activity

              People

              • Assignee:
                vitalii Vitalii Diravka
                Reporter:
                rkins Rahul Challapalli
                Reviewer:
                Arina Ielchiieva
              • Votes:
                0 Vote for this issue
                Watchers:
                4 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: