[HIVE-11611] A bad performance regression issue with Parquet happens if Hive does not select any columns

• Type: Sub-task
• Status: Reopened
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 2.0.0
• Assignee: Ferdinand Xu
• Reporter: Sergio Peña
• Created: 20/Aug/15 19:07
• Updated: 06/Oct/21 09:22

Description

A possible performance issue may happen with the below code when using a query like this SELECT count(1) FROM parquetTable.

if (!ColumnProjectionUtils.isReadAllColumns(configuration) && !indexColumnsWanted.isEmpty()) {
        MessageType requestedSchemaByUser =
            getSchemaByIndex(tableSchema, columnNamesList, indexColumnsWanted);
        return new ReadContext(requestedSchemaByUser, contextMetadata);
} else {
  return new ReadContext(tableSchema, contextMetadata);
}

If there are not columns nor indexes selected, then the above code will read the full schema from Parquet even if Hive does not do anything with such values.

Issue Links

• is related to: PARQUET-363 Cannot construct empty MessageType for ReadContext.requestedSchema
• is related to: PARQUET-278 enforce non empty group on MessageType level