[KUDU-3211] Add a cluster supported features request

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.13.0
• Component/s: master, supportability
• Assignee: Unassigned
• Reporter: Grant Henke
• Created: 29/Oct/20 20:27
• Updated: 09/Feb/22 19:10

Description

Recently we have come across a few scenarios where it would be useful to make decisions in client integrations (Backup/Restore, Spark, NiFi, Impala) based on the supported features of the target Kudu cluster. This can especially helpful when we want to use new features by default if available but using the new feature requires client/integration logic changes.

Some recent examples:

• Push bloomfilter predicates only if supported
• Use insert ignore operations (vs session based ignore) only if supported

It is technically possible to be optimistic about the support of a feature and try to handle errors in a clever way using the required feature capabilities of the RPCs. However, that can be difficult to express and near impossible if you want to make a decision for multiple requests or based on what all tablet servers support instead of based on a single request to a single tablet server.

Additionally now that we support rolling restart, we can't assume that because a single master or tablet server supports a feature that all servers in the cluster support the feature.

Some thoughts on the feature/implementation:

• This should be a master request in order to prevent needing to talk to all the tablet servers.
• We could leverage server registration requests or heartbeats to aggregate the current state on the leader master.
• We could represent these features as "cluster" level features and indicate that some (union) or all (intersect) of the servers support a given feature.
• If this request/response is not available in a cluster the response would indicate that feature support is unknown and the user can decide how to proceed.
• If we want to support disabling features via runtime flags we will need to ensure we update the master, maybe via heartbeat, with changed support for a running server.

Issue Links

• is related to: KUDU-1563 Add support for INSERT/UPDATE/DELETE IGNORE
• is related to: IMPALA-10465 Improve Kudu DML error logging in Impala