Uploaded image for project: 'IMPALA'
  1. IMPALA
  2. IMPALA-2798

After adding a column to avro table, Impala returns weird result if codegen is enabled.

    XMLWordPrintableJSON

    Details

      Description

      Step to reproduce:
      in Hive
      1. create an partitioned avro table, "avro1"
      2. insert some data with snappy compression
      3. add a column to the table
      4. insert more data including new column
      5. create a second table "avro2" with the old table schema (not include the new added column)
      6. msck repair table "avro2"
      In Impala
      7. invalidate metadata and query table "avro2", got weird data at the end.
      8. disable codegen and query table again, it shows proper result.

      seems like when codegen enabled, it gets row offset wrong from the data with extra column.

      sometimes impalad hits DCHECK and crashes.

      F1222 16:12:23.601328 21930 read-write-util.cc:29] Check failed: shift <= 64 (70 vs. 64) 
      *** Check failure stack trace: ***
          @          0x21d471d  google::LogMessage::Fail()
          @          0x21d6ebf  google::LogMessage::SendToLog()
          @          0x21d4287  google::LogMessage::Flush()
          @          0x21d774d  google::LogMessageFatal::~LogMessageFatal()
          @          0x16ce213  impala::ReadWriteUtil::ReadZLong()
          @     0x7f00f3f96593  (unknown)
      

        Attachments

          Activity

            People

            • Assignee:
              jyu@cloudera.com Juan Yu
              Reporter:
              jyu@cloudera.com Juan Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: