[IMPALA-3745] Corrupt encoded values in parquet files can cause crashes

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: Impala 2.7.0
• Fix Version/s: Impala 2.7.0, Impala 2.5.2, Impala 2.6.2
• Component/s: Backend
• Labels: crash
• Target Version: Impala 2.7.0, Impala 2.5.2, Impala 2.6.2
• Assignee: Tim Armstrong
• Reporter: Tim Armstrong
• Created: 14/Jun/16 22:10
• Updated: 01/Jun/17 15:04
• Resolved: 16/Jun/16 14:53

Description

The parquet scanner does not validate values read in many cases, so it has a number of failure modes.

We don't validate that the string length is non-negative and fits within the buffer:

template<>
inline int ParquetPlainEncoder::Decode(
    uint8_t* buffer, int fixed_len_size, StringValue* v) {
  memcpy(&v->len, buffer, sizeof(int32_t));
  v->ptr = reinterpret_cast<char*>(buffer) + sizeof(int32_t);
  int bytesize = ByteSize(*v);
  if (fixed_len_size > 0) v->len = std::min(v->len, fixed_len_size);
  // we still read bytesize bytes, even if we truncate
  return bytesize;
}

In general, we don't validate when calling ParquetPlainEncoder::Decode() that we're staying within the data page.

Issue Links

• is related to: IMPALA-5407 Crash SIGSEGV in DeepCopyVarlenData while inserting into sequencefile