[IMPALA-5903] Inconsistent specification of result set and result set metadata

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Affects Version/s: Impala 2.10.0
• Fix Version/s: Impala 3.0
• Component/s: Catalog, Frontend
• Labels: incompatibility
• Target Version: Impala 3.0
• Epic Color: ghx-label-8
• Assignee: Zoltán Borók-Nagy
• Reporter: Lars Volker
• Created: 06/Sep/17 17:58
• Updated: 12/Apr/18 15:29
• Resolved: 11/Apr/18 08:58

Description

Some statements that return a result set declare it in Frontend.java, but other do not. Still the queries seem to work. We should take a closer look at how/where the result set metadata is specified and returned to the user, and ideally, make the paths today consistent (or provide a good explanation for the inconsistency).

For example, compute stats does return a result set, but we never declare it as returning a result set.
You can easily test the behavior of GetResultMetadata() by modifying one of our tests under tests/hs2/

Take a look at Frontend.java and search for COMPUTE_STATS. You will see that we incorrectly declare the result set to be empty.

From fe/src/main/java/org/apache/impala/service/Frontend.java at f8e7c31:

 403     } else if (analysis.isComputeStatsStmt()) {
 404       ddl.op_type = TCatalogOpType.DDL;
 405       TDdlExecRequest req = new TDdlExecRequest();
 406       req.setDdl_type(TDdlType.COMPUTE_STATS);
 407       req.setCompute_stats_params(analysis.getComputeStatsStmt().toThrift());
 408       ddl.setDdl_params(req);
 409       metadata.setColumns(Collections.<TColumn>emptyList());
 410     }

L409 shows an empty list for the result set metadata.

However in fe/src/main/java/org/apache/impala/service/CatalogOpExecutor.java we construct a result set:

 472         case UPDATE_STATS:
 473           Preconditions.checkState(params.isSetUpdate_stats_params());
 474           Reference<Long> numUpdatedColumns = new Reference<>(0L);
 475           alterTableUpdateStats(tbl, params.getUpdate_stats_params(), response,
 476               numUpdatedPartitions, numUpdatedColumns);
 477           reloadTableSchema = true;
 478           resultColVal.setString_val("Updated " + numUpdatedPartitions.getRef() +
 479               " partition(s) and " + numUpdatedColumns.getRef() + " column(s).");
 480           setResultSet = true;
 481           break;

This has the potential to break HS2 clients that expect a result set when computing stats. But in practice, compute stats works and returns a result set, so the current code/flow is confusing.

Issue Links

• is related to: IMPALA-5933 Compute incremental stats should always return a result set
• is related to: IMPALA-6775 Inconsistent ALTER TABLE SET LOCATION output format