[NIFI-12016] Improve leniency for bundle compatibility to allow for rolling upgrades

• Type: Improvement
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0-M1, 1.28.0
• Component/s: Core Framework
• Labels: bundles rolling_upgrade
• Assignee: Mark Payne
• Reporter: Mark Payne
• Created: 30/Aug/23 20:32
• Updated: 02/Oct/24 20:04
• Resolved: 14/Sep/23 17:30

Description

When NiFi loads a flow from disk, if the flow indicates that a components exists with version X, but version X of the nar is not available, we automatically change the version to version Y, provided that (a) Version Y is available and (b) Version Y is the only available version.

However, when attempting to join a cluster, we do not allow for this. The exact version of the NAR that is specified in the flow must be available. Otherwise, the node will not join the cluster.

This was done because we did not want to have a case where we have different versions of NiFi running on different nodes in a cluster. If this is the case, then the nodes will behave differently, but also the UI will show that the version is X even though one of the nodes may be running version Y.

This seems reasonable, but it has caused major issues in one respect: it makes it impossible to perform a rolling upgrade of a NiFi cluster. It means that the entire cluster must be stopped and upgraded at the same time.

Additionally, the logic comes with another downside. Because we allow multiple versions of a NAR to be loaded, the following scenario is common:

A cluster is running with one version, say 2.2.0. But the user also has version 2.1.1 of a particular NAR. Now, when the user upgrades to version 2.4.0, the flow says that Processor X should be version 2.2.0. Now, version 2.4.0 is available but 2.2.0 is not. However, we do not use version 2.4.0, because there are two versions available: 2.1.1 and 2.4.0. Instead, we just "ghost" the component, and it's up to the user to notice this and go in and change the version on that component.

This is easily avoided by making a small change to the logic. Instead of finding a compatible version only when there is exactly 1 version available, we should find a compatible version EITHER when exactly 1 version is available, OR when one of the version is equal to the version of the framework. So in the example above, it means that we'd update the version to 2.4.0.

Making these changes should allow for rolling upgrades of NiFi.

Issue Links

• is related to: NIFI-10676 NiFi does not negotiate compatible versions when loading flow from cluster connection response
• supercedes: NIFI-11162 When upgrading, rolling restart nodes to avoid service interruption
• supercedes: NIFI-10676 NiFi does not negotiate compatible versions when loading flow from cluster connection response
• links to: GitHub Pull Request #7662
• links to: GitHub Pull Request #9205