[NIFI-11464] importing a versioned flow with a nested versioned flow shows nested versioned flow with invalid controller services.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.21.0
• Fix Version/s: 2.0.0-M1, 1.22.0
• Component/s: Flow Versioning
• Environment: nifi 1.21.0 and nifi registry 1.21.0 (on ubuntu 20.04)
• Assignee: Bryan Bende
• Reporter: Patrick A. Mol
• Created: 17/Apr/23 18:06
• Updated: 05/Jun/23 14:35
• Resolved: 05/Jun/23 14:35

Description

When a flow (reusable_flow_Q) has controller services inherited from the hierarchy (process group reusables) and a version of the flow is stored, the flow version contains the references to these external controller services (as seen in an exported flow version [see below]).
When this versioned flow is imported in another flow (tenant_flows) the controller services need to be reset to the controller services in the new hierarchy.
When we have a working flow with the nested versioned flow ready in development we can check this flow into version control.
When we then deploy the flow in production, the nested versioned flow shows up with invalid components. It shows the external controller service identifiers as stored in the flow version.
When we then go back to development version of tenant_flows and make a minor change to the nested versioned flow reusable_flow_Q and commit this change to version control.
Due to this version change, we need to also commit the changes for the tenant_flows process group.
When we now go back to production, and import this new version of tenant_flows, the nested versioned flow reusable_flow_Q does not have invalid controller services.

If you have several flows under development using the same reusable components,
you will likely end up with invalid components after import.
Depending on the amount of versioned flows used, it could be a lot of work.
It could also lead to issues when using the ExecuteStateless processor.

Please see attached template nested_version_flow_issue.xml for a starting point to reproduce the issue. It contains the steps.
Screenprints are attached in a zip file show the process and diagnosis.

Controller services identifiers in version 2.

$ fgrep -C 4 reusables_avro reusable_flow_Q.json.pretty 
    "controllerServices": [
      {
        "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
        "instanceIdentifier": "8f647d06-0187-1000-4be9-14a61f55d904",
        "name": "reusables_avro_reader",
        "comments": "",
        "type": "org.apache.nifi.avro.AvroReader",
        "bundle": {
          "group": "org.apache.nifi",
--
      },
      {
        "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
        "instanceIdentifier": "8f64f2c6-0187-1000-7557-ca63c88054dd",
        "name": "reusables_avro_writer",
        "comments": "",
        "type": "org.apache.nifi.avro.AvroRecordSetWriter",
        "bundle": {
          "group": "org.apache.nifi",
$ head -15 reusable_flow_Q-version-2.json.pretty 
{
  "externalControllerServices": {
    "dc884171-4d75-3854-8604-afab91bd0e60": {
      "identifier": "dc884171-4d75-3854-8604-afab91bd0e60",
      "name": "reusables_avro_reader"
    },
    "b512b238-cdee-3642-b5cb-0c98d30dd133": {
      "identifier": "b512b238-cdee-3642-b5cb-0c98d30dd133",
      "name": "reusables_avro_writer"
    }
  },
  "flowContents": {
    "comments": "used to perform Q ...",
    "componentType": "PROCESS_GROUP",
    "connections": [
 

Controller services identifiers with version 3 committed in process group "tenant_flows".

pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ fgrep -C 4 tenant_flow_avro tenant_flows-version-1.json.pretty 
        ],
        "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d",
        "identifier": "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22",
        "instanceIdentifier": "8f6cb319-0187-1000-b7fa-83340f7055f7",
        "name": "tenant_flow_avro_writer",
        "properties": {
          "compression-format": "NONE",
          "Schema Write Strategy": "avro-embedded",
          "schema-name": "${schema.name}",
--
        ],
        "groupIdentifier": "a984831b-8587-3e17-bbbc-ef4b85c3898d",
        "identifier": "8ff96d88-3dc8-30ed-aeb8-757c26a7b807",
        "instanceIdentifier": "8f6c8a94-0187-1000-af54-2fee12838618",
        "name": "tenant_flow_avro_reader",
        "properties": {
          "schema-name": "${schema.name}",
          "cache-size": "1000",
          "schema-access-strategy": "embedded-avro-schema",
pmo@hpmo:~/Documents.local/nested_versioned_flows_controller_issue$ head -15 reusable_flow_Q-version-3.json.pretty 
{
  "externalControllerServices": {
    "8ff96d88-3dc8-30ed-aeb8-757c26a7b807": {
      "identifier": "8ff96d88-3dc8-30ed-aeb8-757c26a7b807",
      "name": "tenant_flow_avro_reader"
    },
    "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22": {
      "identifier": "5d9df37d-2a52-3f6e-8cd3-3d3ea9550d22",
      "name": "tenant_flow_avro_writer"
    }
  },
  "flowContents": {
    "comments": "used to perform Q ...",
    "componentType": "PROCESS_GROUP",
    "connections": [
 

Issue Links

• links to: GitHub Pull Request #7218