Uploaded image for project: 'Apache NiFi'
  1. Apache NiFi
  2. NIFI-3380

Multiple Versions of the Same Component

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.2.0
    • Component/s: Core Framework
    • Labels:
      None

      Description

      This ticket is to track the work for supporting multiple versions of the same component within NiFi. The overall design for this feature is described in detail at the following wiki page:

      https://cwiki.apache.org/confluence/display/NIFI/Multiple+Versions+of+the+Same+Extension

      This ticket will track only the core NiFi work, and a separate ticket will be created to track enhancements for the NAR Maven Plugin.

      1. nifi-example-processors-nar-1.0.nar
        4.21 MB
        Bryan Bende
      2. nifi-example-processors-nar-2.0.nar
        4.21 MB
        Bryan Bende
      3. nifi-example-service-api-nar-1.0.nar
        4 kB
        Bryan Bende
      4. nifi-example-service-api-nar-2.0.nar
        4 kB
        Bryan Bende
      5. nifi-example-service-nar-1.0.nar
        4.21 MB
        Bryan Bende
      6. nifi-example-service-nar-1.1.nar
        4.21 MB
        Bryan Bende
      7. nifi-example-service-nar-2.0.nar
        4.21 MB
        Bryan Bende

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user bbende opened a pull request:

          https://github.com/apache/nifi/pull/1585

          NIFI-3380 Support Multiple Versions of the Same Component

          This PR introduces support for running multiple versions of the same component.

          In order to make full use of this capability we need to release and updated version of the NAR Maven plugin (currently 1.1.0, need to release 1.2.0) and then update NiFi's master branch to use this new version. In this PR we have left it using the 1.1.0 NAR plugin, and NiFi is able to run as it normally does, but you can't actually run multiple versions of the same component yet because the framework needs the new information from the new NAR plugin.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/bbende/nifi NIFI-3380

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/1585.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1585


          commit f965f87f3cd85ac160218f0308e4651d1e4363e0
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-01-23T15:33:12Z

          NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs

          • Refactoring NarDetails to include all info from MANIFEST
          • Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
          • Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
          • Refactoring bundle API to classes and creating BundleCoordinate
          • Updating FlowController to use BundleCoordinate

          commit 98216f00ec8ea0e8e113d7b1a5ad75db84b38bd6
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-01-24T18:58:57Z

          NIFI-3380: - Updating the UI and DTO model to support showing bundle details that loaded an extension type.

          • Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
          • Updating the formating of the bundle coordinates.
          • Addressing text overflow in the configuration/details dialog.
          • Fixing self referencing functions.
          • Updating extension UI mapping to incorporate bundle coordinates.
          • Discovering custom UIs through the supplied bundles.
          • Adding verification methods for creating extensions through the rest api.
          • Only returning extensions that are common amongst all nodes.
          • Rendering the ghost processors using a dotted border.
          • Adding bundle details to the flow.xml.
          • Loading NiFi build and version details from the framework NAR.
          • Removing properties for build and version details.
          • Wiring together front end and back end changes.
          • Including bundle coordinates in the component data model.
          • Wiring together component data model and flow.xml.
          • Addressing issue when resolve unvesioned dependent NARs.

          commit 95cf4007b6c16f73305fad9f2ff8ed66b0794b0d
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-01-31T17:05:40Z

          NIFI-3380 Updating unit tests to pass based on framework changes

          • Fixing logging of extension types during start up

          commit e1782b13c71b7897a7b5105012fd2a687c9c9961
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-02-02T21:09:45Z

          NIFI-3380: - Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

          commit 4b188857f59fe85ca38303b0b20448867072e18d
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-02-03T14:39:38Z

          NIFI-3380 Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider

          • Adding ability to change processor type at runtime
          • Adding backend code to change type for controller services

          commit d71ba9d33c811d0eee3a29378c7048b5443957aa
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-02-09T18:58:59Z

          NIFI-3380:

          • Cleaning up instance classloaders for temp components.
          • Creating a dialog for changing the version of a component.
          • Updating the formatting of the component type and bundle throughout.
          • Updating the new component dialogs to support selecting source group.
          • Cleaning up new component dialogs.
          • Cleaning up documentation in the cluster node endpoint.

          commit 8892c4a64f861ac98c6cac9370298cda444e0da8
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-02-09T20:07:20Z

          NIFI-3380 Adding missing include in nifi-web-ui pom compressor plugin

          • Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
          • Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework

          commit 854bed6ebe89b871d461a57fc6894cde06523d66
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-02-13T17:53:27Z

          NIFI-3380: - Finishing clean up following rebase.

          commit 6de6e1d615bed4ebb97c0ebb0051a0006f13f5ea
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-02-15T21:22:49Z

          NIFI-3380 Calling lifecycle methods for add and remove when changing versions of a component

          • Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests

          commit 4aecafa2647c2ab4601461f948f33c5c8a74793c
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-02-16T20:19:50Z

          NIFI-3380:

          • Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

          This closes #5.

          commit ac6af1ffa8a35a3a237fcdf0435d8bfc457ee530
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-02-21T22:16:37Z

          NIFI-3380 Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController

          • Adding validation to ensure referenced controller services implement the required API
          • Fixing template instantiation to look up compatible bundle

          commit 15abe12c6c6633ff80b1e639b2c85c3dc1005825
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-02-23T20:19:33Z

          NIFI-3380: - Requiring services/reporting tasks to be disabled/stopped.

          • Only supporting a change version option when the item has multiple versions available.
          • Limiting the possible new controller services to the applicable API version.
          • Showing the implemented API versions for Controller Services.
          • Updating the property descriptor tooltip to indicate the required service requirements.
          • Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
          • Addressing remainder of the issues from recent rebase.

          commit 4b89349a30632d6dbfc270559da06bea65a88edc
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-02-28T21:58:10Z

          NIFI-3380 Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting

          • Refactoring the way missing bundles work to retain the desired bundle if available
          • Fixing logger.isDebugEnabled to be logger.isTraceEnabled

          commit 6095228ff0c5869eb4103ab3ea61a6782ae236e9
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-03-03T15:50:33Z

          NIFI-3380: - Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

          commit 7232b5c4303b6659f4d1d9ea8b617ba54aa6a07a
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-03-07T21:46:41Z

          NIFI-3380 Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

          commit bcd2085b717775490dadbf3fa5bc5652333e4555
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-03-08T22:37:17Z

          NIFI-3380: - Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

          commit 9ecc40a3b52fa6344ad9e1fd218d128950ca6b43
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-03-09T15:49:45Z

          NIFI-3380 Refactoring controller service invocation handler to allow updating the node used by the invocation handler

          • Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
          • Merging whether multiple versions of the component are available
          • Setting NAR plugin back to current released version
          • Cleaning up DocGenerator to not process multiple times

          commit 9b6eb51daeff03d17f7be505f6aa8c2700f3925e
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-03-10T17:38:21Z

          NIFI-3380: - Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

          commit e9e754e38b9384c699b999da8e4812130503b7d0
          Author: Bryan Bende <bbende@apache.org>
          Date: 2017-03-10T19:53:13Z

          NIFI-3380 Improving error messages when looking for bundles


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user bbende opened a pull request: https://github.com/apache/nifi/pull/1585 NIFI-3380 Support Multiple Versions of the Same Component This PR introduces support for running multiple versions of the same component. In order to make full use of this capability we need to release and updated version of the NAR Maven plugin (currently 1.1.0, need to release 1.2.0) and then update NiFi's master branch to use this new version. In this PR we have left it using the 1.1.0 NAR plugin, and NiFi is able to run as it normally does, but you can't actually run multiple versions of the same component yet because the framework needs the new information from the new NAR plugin. You can merge this pull request into a Git repository by running: $ git pull https://github.com/bbende/nifi NIFI-3380 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1585.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1585 commit f965f87f3cd85ac160218f0308e4651d1e4363e0 Author: Bryan Bende <bbende@apache.org> Date: 2017-01-23T15:33:12Z NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs Refactoring NarDetails to include all info from MANIFEST Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API Refactoring bundle API to classes and creating BundleCoordinate Updating FlowController to use BundleCoordinate commit 98216f00ec8ea0e8e113d7b1a5ad75db84b38bd6 Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-01-24T18:58:57Z NIFI-3380 : - Updating the UI and DTO model to support showing bundle details that loaded an extension type. Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs. Updating the formating of the bundle coordinates. Addressing text overflow in the configuration/details dialog. Fixing self referencing functions. Updating extension UI mapping to incorporate bundle coordinates. Discovering custom UIs through the supplied bundles. Adding verification methods for creating extensions through the rest api. Only returning extensions that are common amongst all nodes. Rendering the ghost processors using a dotted border. Adding bundle details to the flow.xml. Loading NiFi build and version details from the framework NAR. Removing properties for build and version details. Wiring together front end and back end changes. Including bundle coordinates in the component data model. Wiring together component data model and flow.xml. Addressing issue when resolve unvesioned dependent NARs. commit 95cf4007b6c16f73305fad9f2ff8ed66b0794b0d Author: Bryan Bende <bbende@apache.org> Date: 2017-01-31T17:05:40Z NIFI-3380 Updating unit tests to pass based on framework changes Fixing logging of extension types during start up commit e1782b13c71b7897a7b5105012fd2a687c9c9961 Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-02-02T21:09:45Z NIFI-3380 : - Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs. commit 4b188857f59fe85ca38303b0b20448867072e18d Author: Bryan Bende <bbende@apache.org> Date: 2017-02-03T14:39:38Z NIFI-3380 Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider Adding ability to change processor type at runtime Adding backend code to change type for controller services commit d71ba9d33c811d0eee3a29378c7048b5443957aa Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-02-09T18:58:59Z NIFI-3380 : Cleaning up instance classloaders for temp components. Creating a dialog for changing the version of a component. Updating the formatting of the component type and bundle throughout. Updating the new component dialogs to support selecting source group. Cleaning up new component dialogs. Cleaning up documentation in the cluster node endpoint. commit 8892c4a64f861ac98c6cac9370298cda444e0da8 Author: Bryan Bende <bbende@apache.org> Date: 2017-02-09T20:07:20Z NIFI-3380 Adding missing include in nifi-web-ui pom compressor plugin Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework commit 854bed6ebe89b871d461a57fc6894cde06523d66 Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-02-13T17:53:27Z NIFI-3380 : - Finishing clean up following rebase. commit 6de6e1d615bed4ebb97c0ebb0051a0006f13f5ea Author: Bryan Bende <bbende@apache.org> Date: 2017-02-15T21:22:49Z NIFI-3380 Calling lifecycle methods for add and remove when changing versions of a component Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests commit 4aecafa2647c2ab4601461f948f33c5c8a74793c Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-02-16T20:19:50Z NIFI-3380 : Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles. This closes #5. commit ac6af1ffa8a35a3a237fcdf0435d8bfc457ee530 Author: Bryan Bende <bbende@apache.org> Date: 2017-02-21T22:16:37Z NIFI-3380 Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController Adding validation to ensure referenced controller services implement the required API Fixing template instantiation to look up compatible bundle commit 15abe12c6c6633ff80b1e639b2c85c3dc1005825 Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-02-23T20:19:33Z NIFI-3380 : - Requiring services/reporting tasks to be disabled/stopped. Only supporting a change version option when the item has multiple versions available. Limiting the possible new controller services to the applicable API version. Showing the implemented API versions for Controller Services. Updating the property descriptor tooltip to indicate the required service requirements. Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog. Addressing remainder of the issues from recent rebase. commit 4b89349a30632d6dbfc270559da06bea65a88edc Author: Bryan Bende <bbende@apache.org> Date: 2017-02-28T21:58:10Z NIFI-3380 Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting Refactoring the way missing bundles work to retain the desired bundle if available Fixing logger.isDebugEnabled to be logger.isTraceEnabled commit 6095228ff0c5869eb4103ab3ea61a6782ae236e9 Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-03-03T15:50:33Z NIFI-3380 : - Auditing when user changes the bundle. - Ensuring bundle details are present in templates. commit 7232b5c4303b6659f4d1d9ea8b617ba54aa6a07a Author: Bryan Bende <bbende@apache.org> Date: 2017-03-07T21:46:41Z NIFI-3380 Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory commit bcd2085b717775490dadbf3fa5bc5652333e4555 Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-03-08T22:37:17Z NIFI-3380 : - Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone. commit 9ecc40a3b52fa6344ad9e1fd218d128950ca6b43 Author: Bryan Bende <bbende@apache.org> Date: 2017-03-09T15:49:45Z NIFI-3380 Refactoring controller service invocation handler to allow updating the node used by the invocation handler Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow Merging whether multiple versions of the component are available Setting NAR plugin back to current released version Cleaning up DocGenerator to not process multiple times commit 9b6eb51daeff03d17f7be505f6aa8c2700f3925e Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-03-10T17:38:21Z NIFI-3380 : - Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs. commit e9e754e38b9384c699b999da8e4812130503b7d0 Author: Bryan Bende <bbende@apache.org> Date: 2017-03-10T19:53:13Z NIFI-3380 Improving error messages when looking for bundles
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user scottyaslan commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105747481

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-actions.js —
          @@ -1214,6 +1217,23 @@
          },

          /**
          + * Views the state for the specified processor.
          + *
          + * @param

          {selection}

          selection
          + */
          + changeVersion: function (selection) {
          + if (selection.size() !== 1 || !nf.CanvasUtils.isProcessor(selection))

          { + return; + }

          +
          + // get the processor data
          + var processor = selection.datum();
          +
          + // attempt to change the version of the specified component
          + nf.ComponentVersion.promptForVersionChange(processor);
          — End diff –

          We should not be calling anything from the global scope. Instead we should have this module require('nf.ComponentVersion')...

          Show
          githubbot ASF GitHub Bot added a comment - Github user scottyaslan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105747481 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-actions.js — @@ -1214,6 +1217,23 @@ }, /** + * Views the state for the specified processor. + * + * @param {selection} selection + */ + changeVersion: function (selection) { + if (selection.size() !== 1 || !nf.CanvasUtils.isProcessor(selection)) { + return; + } + + // get the processor data + var processor = selection.datum(); + + // attempt to change the version of the specified component + nf.ComponentVersion.promptForVersionChange(processor); — End diff – We should not be calling anything from the global scope. Instead we should have this module require('nf.ComponentVersion')...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user scottyaslan commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105746551

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-bootstrap.js —
          @@ -339,6 +342,7 @@
          nfActions.init();
          nfQueueListing.init();
          nfComponentState.init();
          + nfComponentVersion.init(nfSettings, nfProcessGroupConfiguration);
          — End diff –

          Should be able to remove the injection of the nfProcessGroupConfiguration module here.

          Show
          githubbot ASF GitHub Bot added a comment - Github user scottyaslan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105746551 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-canvas-bootstrap.js — @@ -339,6 +342,7 @@ nfActions.init(); nfQueueListing.init(); nfComponentState.init(); + nfComponentVersion.init(nfSettings, nfProcessGroupConfiguration); — End diff – Should be able to remove the injection of the nfProcessGroupConfiguration module here.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user scottyaslan commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105747710

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-component-version.js —
          @@ -0,0 +1,340 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +/* global nf */
          +
          +/**
          + * Views state for a given component.
          + */
          +(function (root, factory) {
          + if (typeof define === 'function' && define.amd) {
          + define(['jquery',
          + 'nf.ErrorHandler',
          + 'nf.Common',
          + 'nf.Client',
          + 'nf.CanvasUtils',
          + 'nf.ng.Bridge'],
          — End diff –

          This module should require('nf.ProcessGroupConfiguration')...

          Show
          githubbot ASF GitHub Bot added a comment - Github user scottyaslan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105747710 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-component-version.js — @@ -0,0 +1,340 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* global nf */ + +/** + * Views state for a given component. + */ +(function (root, factory) { + if (typeof define === 'function' && define.amd) { + define(['jquery', + 'nf.ErrorHandler', + 'nf.Common', + 'nf.Client', + 'nf.CanvasUtils', + 'nf.ng.Bridge'], — End diff – This module should require('nf.ProcessGroupConfiguration')...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user scottyaslan commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105746348

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml —
          @@ -440,6 +440,7 @@
          <include>$

          {staging.dir}/js/nf/canvas/nf-clipboard.js</include>
          <include>${staging.dir}

          /js/nf/canvas/nf-custom-ui.js</include>
          <include>$

          {staging.dir}/js/nf/canvas/nf-canvas-utils.js</include>
          + <include>${staging.dir}

          /js/nf/canvas/nf-component-version.js</include>
          — End diff –

          I think we can move this down in the order. If you move it below the nf-process-group-configuration.js. then in the nfProcessGroupConfiguration module you can load nfComponentVersion module. If you do this then you can avoid having to inject the nfProcessGroupConfiguration module into the nfComponentVersion module in the nf-canvas-bootstrap.js.

          Show
          githubbot ASF GitHub Bot added a comment - Github user scottyaslan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105746348 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml — @@ -440,6 +440,7 @@ <include>$ {staging.dir}/js/nf/canvas/nf-clipboard.js</include> <include>${staging.dir} /js/nf/canvas/nf-custom-ui.js</include> <include>$ {staging.dir}/js/nf/canvas/nf-canvas-utils.js</include> + <include>${staging.dir} /js/nf/canvas/nf-component-version.js</include> — End diff – I think we can move this down in the order. If you move it below the nf-process-group-configuration.js. then in the nfProcessGroupConfiguration module you can load nfComponentVersion module. If you do this then you can avoid having to inject the nfProcessGroupConfiguration module into the nfComponentVersion module in the nf-canvas-bootstrap.js.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user scottyaslan commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105746988

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-component-version.js —
          @@ -0,0 +1,340 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +
          +/* global nf */
          +
          +/**
          + * Views state for a given component.
          + */
          +(function (root, factory) {
          + if (typeof define === 'function' && define.amd) {
          + define(['jquery',
          + 'nf.ErrorHandler',
          + 'nf.Common',
          + 'nf.Client',
          + 'nf.CanvasUtils',
          + 'nf.ng.Bridge'],
          + function ($, Slick, nfErrorHandler, nfCommon, nfClient, nfCanvasUtils, nfNgBridge)

          { + return (nf.ComponentState = factory($, nfErrorHandler, nfCommon, nfClient, nfCanvasUtils, nfNgBridge)); + }

          );
          + } else if (typeof exports === 'object' && typeof module === 'object')

          { + module.exports = (nf.ComponentState = + factory(require('jquery'), + require('nf.ErrorHandler'), + require('nf.Common', + require('nf.Client'), + require('nf.CanvasUtils'), + require('nf.ng.Bridge')))); + }

          else

          { + nf.ComponentVersion = factory(root.$, + root.nf.ErrorHandler, + root.nf.Common, + root.nf.Client, + root.nf.CanvasUtils, + root.nf.ng.Bridge); + }

          +}(this, function ($, nfErrorHandler, nfCommon, nfClient, nfCanvasUtils, nfNgBridge) {
          + 'use strict';
          +
          + var versionMap;
          + var nfSettings;
          + var nfProcessGroupConfiguration;
          +
          + /**
          + * Gets the URI for retrieving available types/bundles
          + *
          + * @param

          {object} componentEntity
          + * @returns {string} uri
          + */
          + var getTypeUri = function (componentEntity) {
          + if (componentEntity.type === 'ReportingTask') { + return '../nifi-api/flow/reporting-task-types'; + } else if (componentEntity.type === 'ControllerService') { + return '../nifi-api/flow/controller-service-types'; + } else { + return '../nifi-api/flow/processor-types'; + }
          + };
          +
          + /**
          + * Gets the field to use to access the returned types/bundles.
          + *
          + * @param {object}

          componentEntity
          + * @returns

          {string}

          field
          + */
          + var getTypeField = function (componentEntity) {
          + if (componentEntity.type === 'ReportingTask')

          { + return 'reportingTaskTypes'; + }

          else if (componentEntity.type === 'ControllerService')

          { + return 'controllerServiceTypes'; + }

          else

          { + return 'processorTypes'; + }

          + };
          +
          + /**
          + * Reset the dialog.
          + */
          + var resetDialog = function () {
          + // clear the versions
          + var versions = versionMap.keys();
          + $.each(versions, function (_, version)

          { + versionMap.remove(version); + }

          );
          +
          + // clear the service apis
          + $('#component-version-controller-service-apis').empty();
          + $('#component-version-controller-service-apis-container').hide();
          +
          + // clear the fields
          + $('#component-version-name').text('');
          + $('#component-version-bundle').text('');
          + $('#component-version-tags').text('');
          + $('#component-version-restriction').removeClass('unset').text('');
          + $('#component-version-description').text('');
          +
          + // destroy the version combo
          + $('#component-version-selector').combo('destroy');
          +
          + // removed the stored data
          + $('#component-version-dialog').removeData('component');
          + };
          +
          + /**
          + * Sets the specified option.
          + *
          + * @param

          {object}

          selectedOption
          + */
          + var select = function (selectedOption) {
          + var documentedType = versionMap.get(selectedOption.value);
          +
          + // set any restriction
          + if (nfCommon.isDefinedAndNotNull(documentedType.usageRestriction))

          { + $('#component-version-restriction').text(documentedType.usageRestriction); + }

          else

          { + $('#component-version-restriction').addClass('unset').text('No restriction'); + }

          +
          + // update the service apis if necessary
          + if (!nfCommon.isEmpty(documentedType.controllerServiceApis))

          { + var formattedControllerServiceApis = nfCommon.getFormattedServiceApis(documentedType.controllerServiceApis); + var serviceTips = nfCommon.formatUnorderedList(formattedControllerServiceApis); + $('#component-version-controller-service-apis').empty().append(serviceTips); + $('#component-version-controller-service-apis-container').show(); + }

          +
          + // update the tags and description
          + $('#component-version-tags').text(documentedType.tags.join(', '));
          + $('#component-version-description').text(documentedType.description);
          + };
          +
          + return {
          + init: function (settings, processGroupConfiguration) {
          + versionMap = d3.map();
          + nfSettings = settings;
          + nfProcessGroupConfiguration = processGroupConfiguration;
          — End diff –

          Should be able to remove the injection of the nfProcessGroupConfiguration module here too...

          Show
          githubbot ASF GitHub Bot added a comment - Github user scottyaslan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105746988 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-component-version.js — @@ -0,0 +1,340 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +/* global nf */ + +/** + * Views state for a given component. + */ +(function (root, factory) { + if (typeof define === 'function' && define.amd) { + define(['jquery', + 'nf.ErrorHandler', + 'nf.Common', + 'nf.Client', + 'nf.CanvasUtils', + 'nf.ng.Bridge'], + function ($, Slick, nfErrorHandler, nfCommon, nfClient, nfCanvasUtils, nfNgBridge) { + return (nf.ComponentState = factory($, nfErrorHandler, nfCommon, nfClient, nfCanvasUtils, nfNgBridge)); + } ); + } else if (typeof exports === 'object' && typeof module === 'object') { + module.exports = (nf.ComponentState = + factory(require('jquery'), + require('nf.ErrorHandler'), + require('nf.Common', + require('nf.Client'), + require('nf.CanvasUtils'), + require('nf.ng.Bridge')))); + } else { + nf.ComponentVersion = factory(root.$, + root.nf.ErrorHandler, + root.nf.Common, + root.nf.Client, + root.nf.CanvasUtils, + root.nf.ng.Bridge); + } +}(this, function ($, nfErrorHandler, nfCommon, nfClient, nfCanvasUtils, nfNgBridge) { + 'use strict'; + + var versionMap; + var nfSettings; + var nfProcessGroupConfiguration; + + /** + * Gets the URI for retrieving available types/bundles + * + * @param {object} componentEntity + * @returns {string} uri + */ + var getTypeUri = function (componentEntity) { + if (componentEntity.type === 'ReportingTask') { + return '../nifi-api/flow/reporting-task-types'; + } else if (componentEntity.type === 'ControllerService') { + return '../nifi-api/flow/controller-service-types'; + } else { + return '../nifi-api/flow/processor-types'; + } + }; + + /** + * Gets the field to use to access the returned types/bundles. + * + * @param {object} componentEntity + * @returns {string} field + */ + var getTypeField = function (componentEntity) { + if (componentEntity.type === 'ReportingTask') { + return 'reportingTaskTypes'; + } else if (componentEntity.type === 'ControllerService') { + return 'controllerServiceTypes'; + } else { + return 'processorTypes'; + } + }; + + /** + * Reset the dialog. + */ + var resetDialog = function () { + // clear the versions + var versions = versionMap.keys(); + $.each(versions, function (_, version) { + versionMap.remove(version); + } ); + + // clear the service apis + $('#component-version-controller-service-apis').empty(); + $('#component-version-controller-service-apis-container').hide(); + + // clear the fields + $('#component-version-name').text(''); + $('#component-version-bundle').text(''); + $('#component-version-tags').text(''); + $('#component-version-restriction').removeClass('unset').text(''); + $('#component-version-description').text(''); + + // destroy the version combo + $('#component-version-selector').combo('destroy'); + + // removed the stored data + $('#component-version-dialog').removeData('component'); + }; + + /** + * Sets the specified option. + * + * @param {object} selectedOption + */ + var select = function (selectedOption) { + var documentedType = versionMap.get(selectedOption.value); + + // set any restriction + if (nfCommon.isDefinedAndNotNull(documentedType.usageRestriction)) { + $('#component-version-restriction').text(documentedType.usageRestriction); + } else { + $('#component-version-restriction').addClass('unset').text('No restriction'); + } + + // update the service apis if necessary + if (!nfCommon.isEmpty(documentedType.controllerServiceApis)) { + var formattedControllerServiceApis = nfCommon.getFormattedServiceApis(documentedType.controllerServiceApis); + var serviceTips = nfCommon.formatUnorderedList(formattedControllerServiceApis); + $('#component-version-controller-service-apis').empty().append(serviceTips); + $('#component-version-controller-service-apis-container').show(); + } + + // update the tags and description + $('#component-version-tags').text(documentedType.tags.join(', ')); + $('#component-version-description').text(documentedType.description); + }; + + return { + init: function (settings, processGroupConfiguration) { + versionMap = d3.map(); + nfSettings = settings; + nfProcessGroupConfiguration = processGroupConfiguration; — End diff – Should be able to remove the injection of the nfProcessGroupConfiguration module here too...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mcgilman commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105758869

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml —
          @@ -440,6 +440,7 @@
          <include>$

          {staging.dir}/js/nf/canvas/nf-clipboard.js</include>
          <include>${staging.dir}

          /js/nf/canvas/nf-custom-ui.js</include>
          <include>$

          {staging.dir}/js/nf/canvas/nf-canvas-utils.js</include>
          + <include>${staging.dir}

          /js/nf/canvas/nf-component-version.js</include>
          — End diff –

          Got it. I feel like it wasn't possible when I originally typed it up. However, this module went through a couple iterations during the development of this feature. I'll try updating the dependency injection with your suggestions now.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mcgilman commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105758869 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/pom.xml — @@ -440,6 +440,7 @@ <include>$ {staging.dir}/js/nf/canvas/nf-clipboard.js</include> <include>${staging.dir} /js/nf/canvas/nf-custom-ui.js</include> <include>$ {staging.dir}/js/nf/canvas/nf-canvas-utils.js</include> + <include>${staging.dir} /js/nf/canvas/nf-component-version.js</include> — End diff – Got it. I feel like it wasn't possible when I originally typed it up. However, this module went through a couple iterations during the development of this feature. I'll try updating the dependency injection with your suggestions now.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mcgilman commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105759544

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-actions.js —
          @@ -1214,6 +1217,23 @@
          },

          /**
          + * Views the state for the specified processor.
          + *
          + * @param

          {selection}

          selection
          + */
          + changeVersion: function (selection) {
          + if (selection.size() !== 1 || !nf.CanvasUtils.isProcessor(selection))

          { + return; + }

          +
          + // get the processor data
          + var processor = selection.datum();
          +
          + // attempt to change the version of the specified component
          + nf.ComponentVersion.promptForVersionChange(processor);
          — End diff –

          Yup. Must have missed this one. Much of this was written prior to getting the modularization effort in. I went back through and updated each invocation using `nf.`. I'll update this one and make sure I didn't miss any others.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mcgilman commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105759544 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/nf/canvas/nf-actions.js — @@ -1214,6 +1217,23 @@ }, /** + * Views the state for the specified processor. + * + * @param {selection} selection + */ + changeVersion: function (selection) { + if (selection.size() !== 1 || !nf.CanvasUtils.isProcessor(selection)) { + return; + } + + // get the processor data + var processor = selection.datum(); + + // attempt to change the version of the specified component + nf.ComponentVersion.promptForVersionChange(processor); — End diff – Yup. Must have missed this one. Much of this was written prior to getting the modularization effort in. I went back through and updated each invocation using `nf.`. I'll update this one and make sure I didn't miss any others.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user scottyaslan commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105973908

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/resources/filters/canvas.properties —
          @@ -28,6 +28,7 @@ nf.canvas.script.tags=<script type="text/javascript" src="js/nf/nf-ng-bridge.js?
          <script type="text/javascript" src="js/nf/canvas/nf-clipboard.js?$

          {project.version}"></script>\n\
          <script type="text/javascript" src="js/nf/canvas/nf-custom-ui.js?${project.version}

          "></script>\n\
          <script type="text/javascript" src="js/nf/canvas/nf-canvas-utils.js?$

          {project.version}"></script>\n\
          +<script type="text/javascript" src="js/nf/canvas/nf-component-version.js?${project.version}

          "></script>\n\
          — End diff –

          Will need to update the load order here as well...

          Show
          githubbot ASF GitHub Bot added a comment - Github user scottyaslan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105973908 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/resources/filters/canvas.properties — @@ -28,6 +28,7 @@ nf.canvas.script.tags=<script type="text/javascript" src="js/nf/nf-ng-bridge.js? <script type="text/javascript" src="js/nf/canvas/nf-clipboard.js?$ {project.version}"></script>\n\ <script type="text/javascript" src="js/nf/canvas/nf-custom-ui.js?${project.version} "></script>\n\ <script type="text/javascript" src="js/nf/canvas/nf-canvas-utils.js?$ {project.version}"></script>\n\ +<script type="text/javascript" src="js/nf/canvas/nf-component-version.js?${project.version} "></script>\n\ — End diff – Will need to update the load order here as well...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user scottyaslan commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105977072

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js —
          @@ -939,27 +939,30 @@
          /**

          • Gets the available controller services that implement the specified type and
          • prompts the user to create one.
              • End diff –

          Why did you delete the param docs?

          Show
          githubbot ASF GitHub Bot added a comment - Github user scottyaslan commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105977072 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js — @@ -939,27 +939,30 @@ /** Gets the available controller services that implement the specified type and prompts the user to create one. End diff – Why did you delete the param docs?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mcgilman commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105980001

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js —
          @@ -939,27 +939,30 @@
          /**

          • Gets the available controller services that implement the specified type and
          • prompts the user to create one.
              • End diff –

          Yeah, werid... Will restore.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mcgilman commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105980001 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/webapp/js/jquery/propertytable/jquery.propertytable.js — @@ -939,27 +939,30 @@ /** Gets the available controller services that implement the specified type and prompts the user to create one. End diff – Yeah, werid... Will restore.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mcgilman commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r105980032

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/resources/filters/canvas.properties —
          @@ -28,6 +28,7 @@ nf.canvas.script.tags=<script type="text/javascript" src="js/nf/nf-ng-bridge.js?
          <script type="text/javascript" src="js/nf/canvas/nf-clipboard.js?$

          {project.version}"></script>\n\
          <script type="text/javascript" src="js/nf/canvas/nf-custom-ui.js?${project.version}

          "></script>\n\
          <script type="text/javascript" src="js/nf/canvas/nf-canvas-utils.js?$

          {project.version}"></script>\n\
          +<script type="text/javascript" src="js/nf/canvas/nf-component-version.js?${project.version}

          "></script>\n\
          — End diff –

          Yup.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mcgilman commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r105980032 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-ui/src/main/resources/filters/canvas.properties — @@ -28,6 +28,7 @@ nf.canvas.script.tags=<script type="text/javascript" src="js/nf/nf-ng-bridge.js? <script type="text/javascript" src="js/nf/canvas/nf-clipboard.js?$ {project.version}"></script>\n\ <script type="text/javascript" src="js/nf/canvas/nf-custom-ui.js?${project.version} "></script>\n\ <script type="text/javascript" src="js/nf/canvas/nf-canvas-utils.js?$ {project.version}"></script>\n\ +<script type="text/javascript" src="js/nf/canvas/nf-component-version.js?${project.version} "></script>\n\ — End diff – Yup.
          Hide
          bende Bryan Bende added a comment -

          Uploading NARs that can be used for testing. MyProcessor v1 depends on MyService v1.0 which has two implementations - StandardMyService v1.0 and StandardMyService v1.1. MyProcessor v2 depends on MyService v2.0 and there is one implementation - StandardMyService 2.0.

          Show
          bende Bryan Bende added a comment - Uploading NARs that can be used for testing. MyProcessor v1 depends on MyService v1.0 which has two implementations - StandardMyService v1.0 and StandardMyService v1.1. MyProcessor v2 depends on MyService v2.0 and there is one implementation - StandardMyService 2.0.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r106502862

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java —
          @@ -857,7 +858,13 @@ public Relationship getRelationship(final String relationshipName) {

          @Override
          public Processor getProcessor()

          { - return this.processor; + return processorRef.get().getProcessor(); + }

          +
          + @Override
          + public void setProcessor(final LoggableComponent<Processor> processor) {
          + final ProcessorDetails processorDetails = new ProcessorDetails(processor);
          — End diff –

          We should probably have some thread protection here to make sure that the Processor is indeed stopped. This would mean that we need to enter a synchronized block here, then verify that the processor is not running (!isRunning()) and then do the processorRef.set(processorDetails). We would also need to ensure that in the start() method that the act of changing the scheduledState from STOPPED to STARTED is also synchronized to ensure atomicity.

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r106502862 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java — @@ -857,7 +858,13 @@ public Relationship getRelationship(final String relationshipName) { @Override public Processor getProcessor() { - return this.processor; + return processorRef.get().getProcessor(); + } + + @Override + public void setProcessor(final LoggableComponent<Processor> processor) { + final ProcessorDetails processorDetails = new ProcessorDetails(processor); — End diff – We should probably have some thread protection here to make sure that the Processor is indeed stopped. This would mean that we need to enter a synchronized block here, then verify that the processor is not running (!isRunning()) and then do the processorRef.set(processorDetails). We would also need to ensure that in the start() method that the act of changing the scheduledState from STOPPED to STARTED is also synchronized to ensure atomicity.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r106504089

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java —
          @@ -1457,4 +1466,85 @@ public String getProcessGroupIdentifier()

          { final ProcessGroup group = getProcessGroup(); return group == null ? null : group.getIdentifier(); }

          +
          + private static final class ProcessorDetails {
          — End diff –

          Would recommend that this be externalized into its own class file, since this class already is at nearly 1500 lines of code, and this inner class is static. Not a big deal but worth considering.

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r106504089 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java — @@ -1457,4 +1466,85 @@ public String getProcessGroupIdentifier() { final ProcessGroup group = getProcessGroup(); return group == null ? null : group.getIdentifier(); } + + private static final class ProcessorDetails { — End diff – Would recommend that this be externalized into its own class file, since this class already is at nearly 1500 lines of code, and this inner class is static. Not a big deal but worth considering.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r106505581

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java —
          @@ -1457,4 +1466,85 @@ public String getProcessGroupIdentifier()

          { final ProcessGroup group = getProcessGroup(); return group == null ? null : group.getIdentifier(); }

          +
          + private static final class ProcessorDetails {
          — End diff –

          Similarly, for ControllerServiceDetails and ReportingTaskDetails

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r106505581 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardProcessorNode.java — @@ -1457,4 +1466,85 @@ public String getProcessGroupIdentifier() { final ProcessGroup group = getProcessGroup(); return group == null ? null : group.getIdentifier(); } + + private static final class ProcessorDetails { — End diff – Similarly, for ControllerServiceDetails and ReportingTaskDetails
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r106506092

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/reporting/AbstractReportingTaskNode.java —
          @@ -125,7 +141,12 @@ public void setSchedulingPeriod(final String schedulingPeriod) {

          @Override
          public ReportingTask getReportingTask()

          { - return reportingTask; + return reportingTaskRef.get().getReportingTask(); + }

          +
          + @Override
          + public void setReportingTask(final LoggableComponent<ReportingTask> reportingTask) {
          + this.reportingTaskRef.set(new ReportingTaskDetails(reportingTask));
          — End diff –

          We should make sure here that the Reporting Task is stopped, similar to Processors. And should also ensure that Controller Services are disabled when calling setControllerService.

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r106506092 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/reporting/AbstractReportingTaskNode.java — @@ -125,7 +141,12 @@ public void setSchedulingPeriod(final String schedulingPeriod) { @Override public ReportingTask getReportingTask() { - return reportingTask; + return reportingTaskRef.get().getReportingTask(); + } + + @Override + public void setReportingTask(final LoggableComponent<ReportingTask> reportingTask) { + this.reportingTaskRef.set(new ReportingTaskDetails(reportingTask)); — End diff – We should make sure here that the Reporting Task is stopped, similar to Processors. And should also ensure that Controller Services are disabled when calling setControllerService.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1585#discussion_r106662774

          — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardFlowService.java —
          @@ -905,6 +932,14 @@ private void loadFromConnectionResponse(final ConnectionResponse response) throw

          }

          + private void initialzeController() throws IOException {
          — End diff –

          Typo in method name - initialze should be initialize

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1585#discussion_r106662774 — Diff: nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/StandardFlowService.java — @@ -905,6 +932,14 @@ private void loadFromConnectionResponse(final ConnectionResponse response) throw } + private void initialzeController() throws IOException { — End diff – Typo in method name - initialze should be initialize
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 commented on the issue:

          https://github.com/apache/nifi/pull/1585

          @bbende I created a 3-node cluster. I added MyProcessor v2.0 to the flow. I then shut down node 3, removed the nifi-example-processors-nar-2.0.nar file from the lib/ directory and removed conf/flow.xml.gz. I then restarted Node 3. It fails to startup because the cluster flow cannot be inherited, due to the node not having v2.0 of the Processor. This, I think, is the expected behavior. However, it fails with the following error message in the logs:

          ```
          2017-03-17 13:14:58,602 ERROR [main] o.a.nifi.controller.StandardFlowService Failed to load flow from cluster due to: org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow.
          org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow.
          at org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:921) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:513) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:780) [nifi-jetty-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.NiFi.<init>(NiFi.java:162) [nifi-runtime-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.NiFi.main(NiFi.java:269) [nifi-runtime-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          Caused by: org.apache.nifi.controller.UninheritableFlowException: org.apache.nifi.processors.example.MyProcessor from org.apache.nifi:nifi-example-processors-nar:2.0 is not known to this NiFi instance.
          at org.apache.nifi.controller.StandardFlowSynchronizer.checkBundleCompatibility(StandardFlowSynchronizer.java:431) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:246) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1535) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:707) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:896) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          ... 4 common frames omitted
          Caused by: java.lang.IllegalStateException: org.apache.nifi.processors.example.MyProcessor from org.apache.nifi:nifi-example-processors-nar:2.0 is not known to this NiFi instance.
          at org.apache.nifi.util.BundleUtils.findCompatibleBundle(BundleUtils.java:54) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.util.BundleUtils.getBundle(BundleUtils.java:88) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          at org.apache.nifi.controller.StandardFlowSynchronizer.checkBundleCompatibility(StandardFlowSynchronizer.java:429) ~[nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT]
          ... 9 common frames omitted
          ```

          I find this error message confusing. It used to make sense when the local flow as different than the cluster flow and that was the only way we could get into this situation. However, it now is caused by failing to create the necessary Processor. We should use a different error message.

          Either we could throw a different Exception (perhaps something that extends UninheritableFlowException?) and provide a different error message, or we should make the message more generic, such as: "Unable to update local flow to match the cluster's flow" instead of "Local flow is different from cluster flow," given that the local flow was empty and therefore should try to inherit the cluster flow.

          Does that make sense?

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/1585 @bbende I created a 3-node cluster. I added MyProcessor v2.0 to the flow. I then shut down node 3, removed the nifi-example-processors-nar-2.0.nar file from the lib/ directory and removed conf/flow.xml.gz. I then restarted Node 3. It fails to startup because the cluster flow cannot be inherited, due to the node not having v2.0 of the Processor. This, I think, is the expected behavior. However, it fails with the following error message in the logs: ``` 2017-03-17 13:14:58,602 ERROR [main] o.a.nifi.controller.StandardFlowService Failed to load flow from cluster due to: org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow. org.apache.nifi.controller.UninheritableFlowException: Failed to connect node to cluster because local flow is different than cluster flow. at org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:921) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:513) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:780) [nifi-jetty-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.NiFi.<init>(NiFi.java:162) [nifi-runtime-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.NiFi.main(NiFi.java:269) [nifi-runtime-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] Caused by: org.apache.nifi.controller.UninheritableFlowException: org.apache.nifi.processors.example.MyProcessor from org.apache.nifi:nifi-example-processors-nar:2.0 is not known to this NiFi instance. at org.apache.nifi.controller.StandardFlowSynchronizer.checkBundleCompatibility(StandardFlowSynchronizer.java:431) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:246) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1535) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:83) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:707) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.controller.StandardFlowService.loadFromConnectionResponse(StandardFlowService.java:896) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] ... 4 common frames omitted Caused by: java.lang.IllegalStateException: org.apache.nifi.processors.example.MyProcessor from org.apache.nifi:nifi-example-processors-nar:2.0 is not known to this NiFi instance. at org.apache.nifi.util.BundleUtils.findCompatibleBundle(BundleUtils.java:54) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.util.BundleUtils.getBundle(BundleUtils.java:88) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] at org.apache.nifi.controller.StandardFlowSynchronizer.checkBundleCompatibility(StandardFlowSynchronizer.java:429) ~ [nifi-framework-core-1.2.0-SNAPSHOT.jar:1.2.0-SNAPSHOT] ... 9 common frames omitted ``` I find this error message confusing. It used to make sense when the local flow as different than the cluster flow and that was the only way we could get into this situation. However, it now is caused by failing to create the necessary Processor. We should use a different error message. Either we could throw a different Exception (perhaps something that extends UninheritableFlowException?) and provide a different error message, or we should make the message more generic, such as: "Unable to update local flow to match the cluster's flow" instead of "Local flow is different from cluster flow," given that the local flow was empty and therefore should try to inherit the cluster flow. Does that make sense?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bbende commented on the issue:

          https://github.com/apache/nifi/pull/1585

          @markap14 The PR has been updated to address your comments and to fix a couple of other issues that were identified, also rebased against master as of yesterday evening.

          Here are the changes since your last review:

          I'm going to continue testing today, but let me know if there is anything else to address.

          Show
          githubbot ASF GitHub Bot added a comment - Github user bbende commented on the issue: https://github.com/apache/nifi/pull/1585 @markap14 The PR has been updated to address your comments and to fix a couple of other issues that were identified, also rebased against master as of yesterday evening. Here are the changes since your last review: https://github.com/apache/nifi/pull/1585/commits/fadf43d7ccc755b5a5e3b52692e6b8f814afb657 (synchronization in node classes) https://github.com/apache/nifi/pull/1585/commits/b1ed76c2bc4ec3a639d360017fe899a8fd5777a7 (CS bundled with API + restructure Hive NAR) https://github.com/apache/nifi/pull/1585/commits/6b2ac1fb77ee08ce4b183e3b78c7c178c38256ae (missing components in fingerprinting) https://github.com/apache/nifi/pull/1585/commits/8988cd9a88ab528bbbb25c07260b17b5179c5886 (checkstyles and failing tests) I'm going to continue testing today, but let me know if there is anything else to address.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit d90cf846b9dcd49acdeac7bb5fcd13ee80dcdba6 in nifi's branch refs/heads/master from Bryan Bende
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d90cf84 ]

          NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs

          • Refactoring NarDetails to include all info from MANIFEST
          • Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
          • Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
          • Refactoring bundle API to classes and creating BundleCoordinate
          • Updating FlowController to use BundleCoordinate
          • Updating the UI and DTO model to support showing bundle details that loaded an extension type.
          • Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
          • Updating the formating of the bundle coordinates.
          • Addressing text overflow in the configuration/details dialog.
          • Fixing self referencing functions.
          • Updating extension UI mapping to incorporate bundle coordinates.
          • Discovering custom UIs through the supplied bundles.
          • Adding verification methods for creating extensions through the rest api.
          • Only returning extensions that are common amongst all nodes.
          • Rendering the ghost processors using a dotted border.
          • Adding bundle details to the flow.xml.
          • Loading NiFi build and version details from the framework NAR.
          • Removing properties for build and version details.
          • Wiring together front end and back end changes.
          • Including bundle coordinates in the component data model.
          • Wiring together component data model and flow.xml.
          • Addressing issue when resolve unvesioned dependent NARs.

          Updating unit tests to pass based on framework changes

          • Fixing logging of extension types during start up
          • Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

          Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider

          • Adding ability to change processor type at runtime
          • Adding backend code to change type for controller services
          • Cleaning up instance classloaders for temp components.
          • Creating a dialog for changing the version of a component.
          • Updating the formatting of the component type and bundle throughout.
          • Updating the new component dialogs to support selecting source group.
          • Cleaning up new component dialogs.
          • Cleaning up documentation in the cluster node endpoint.

          Adding missing include in nifi-web-ui pom compressor plugin

          • Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
          • Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework
          • Finishing clean up following rebase.

          Calling lifecycle methods for add and remove when changing versions of a component

          • Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests
          • Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

          Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController

          • Adding validation to ensure referenced controller services implement the required API
          • Fixing template instantiation to look up compatible bundle
          • Requiring services/reporting tasks to be disabled/stopped.
          • Only supporting a change version option when the item has multiple versions available.
          • Limiting the possible new controller services to the applicable API version.
          • Showing the implemented API versions for Controller Services.
          • Updating the property descriptor tooltip to indicate the required service requirements.
          • Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
          • Addressing remainder of the issues from recent rebase.

          Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting

          • Refactoring the way missing bundles work to retain the desired bundle if available
          • Fixing logger.isDebugEnabled to be logger.isTraceEnabled
          • Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

          Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

          • Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

          Refactoring controller service invocation handler to allow updating the node used by the invocation handler

          • Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
          • Merging whether multiple versions of the component are available
          • Setting NAR plugin back to current released version
          • Cleaning up DocGenerator to not process multiple times

          Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

          Improving error messages when looking for bundles

          Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.

          Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.

          Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0

          Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster

          NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.

          Fixing checkstyles issue and adding back assume checks to distributed cache server test

          Ensuring new component types are sorted correctly when shown initially.

          This closes #1585.

          Show
          jira-bot ASF subversion and git services added a comment - Commit d90cf846b9dcd49acdeac7bb5fcd13ee80dcdba6 in nifi's branch refs/heads/master from Bryan Bende [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d90cf84 ] NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs Refactoring NarDetails to include all info from MANIFEST Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API Refactoring bundle API to classes and creating BundleCoordinate Updating FlowController to use BundleCoordinate Updating the UI and DTO model to support showing bundle details that loaded an extension type. Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs. Updating the formating of the bundle coordinates. Addressing text overflow in the configuration/details dialog. Fixing self referencing functions. Updating extension UI mapping to incorporate bundle coordinates. Discovering custom UIs through the supplied bundles. Adding verification methods for creating extensions through the rest api. Only returning extensions that are common amongst all nodes. Rendering the ghost processors using a dotted border. Adding bundle details to the flow.xml. Loading NiFi build and version details from the framework NAR. Removing properties for build and version details. Wiring together front end and back end changes. Including bundle coordinates in the component data model. Wiring together component data model and flow.xml. Addressing issue when resolve unvesioned dependent NARs. Updating unit tests to pass based on framework changes Fixing logging of extension types during start up Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs. Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider Adding ability to change processor type at runtime Adding backend code to change type for controller services Cleaning up instance classloaders for temp components. Creating a dialog for changing the version of a component. Updating the formatting of the component type and bundle throughout. Updating the new component dialogs to support selecting source group. Cleaning up new component dialogs. Cleaning up documentation in the cluster node endpoint. Adding missing include in nifi-web-ui pom compressor plugin Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework Finishing clean up following rebase. Calling lifecycle methods for add and remove when changing versions of a component Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles. Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController Adding validation to ensure referenced controller services implement the required API Fixing template instantiation to look up compatible bundle Requiring services/reporting tasks to be disabled/stopped. Only supporting a change version option when the item has multiple versions available. Limiting the possible new controller services to the applicable API version. Showing the implemented API versions for Controller Services. Updating the property descriptor tooltip to indicate the required service requirements. Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog. Addressing remainder of the issues from recent rebase. Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting Refactoring the way missing bundles work to retain the desired bundle if available Fixing logger.isDebugEnabled to be logger.isTraceEnabled Auditing when user changes the bundle. - Ensuring bundle details are present in templates. Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone. Refactoring controller service invocation handler to allow updating the node used by the invocation handler Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow Merging whether multiple versions of the component are available Setting NAR plugin back to current released version Cleaning up DocGenerator to not process multiple times Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs. Improving error messages when looking for bundles Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests. Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type. Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0 Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster NIFI-3380 : NIFI-3520 : - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied. Fixing checkstyles issue and adding back assume checks to distributed cache server test Ensuring new component types are sorted correctly when shown initially. This closes #1585.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit d90cf846b9dcd49acdeac7bb5fcd13ee80dcdba6 in nifi's branch refs/heads/master from Bryan Bende
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d90cf84 ]

          NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs

          • Refactoring NarDetails to include all info from MANIFEST
          • Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager
          • Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API
          • Refactoring bundle API to classes and creating BundleCoordinate
          • Updating FlowController to use BundleCoordinate
          • Updating the UI and DTO model to support showing bundle details that loaded an extension type.
          • Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs.
          • Updating the formating of the bundle coordinates.
          • Addressing text overflow in the configuration/details dialog.
          • Fixing self referencing functions.
          • Updating extension UI mapping to incorporate bundle coordinates.
          • Discovering custom UIs through the supplied bundles.
          • Adding verification methods for creating extensions through the rest api.
          • Only returning extensions that are common amongst all nodes.
          • Rendering the ghost processors using a dotted border.
          • Adding bundle details to the flow.xml.
          • Loading NiFi build and version details from the framework NAR.
          • Removing properties for build and version details.
          • Wiring together front end and back end changes.
          • Including bundle coordinates in the component data model.
          • Wiring together component data model and flow.xml.
          • Addressing issue when resolve unvesioned dependent NARs.

          Updating unit tests to pass based on framework changes

          • Fixing logging of extension types during start up
          • Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs.

          Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider

          • Adding ability to change processor type at runtime
          • Adding backend code to change type for controller services
          • Cleaning up instance classloaders for temp components.
          • Creating a dialog for changing the version of a component.
          • Updating the formatting of the component type and bundle throughout.
          • Updating the new component dialogs to support selecting source group.
          • Cleaning up new component dialogs.
          • Cleaning up documentation in the cluster node endpoint.

          Adding missing include in nifi-web-ui pom compressor plugin

          • Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent
          • Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework
          • Finishing clean up following rebase.

          Calling lifecycle methods for add and remove when changing versions of a component

          • Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests
          • Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles.

          Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController

          • Adding validation to ensure referenced controller services implement the required API
          • Fixing template instantiation to look up compatible bundle
          • Requiring services/reporting tasks to be disabled/stopped.
          • Only supporting a change version option when the item has multiple versions available.
          • Limiting the possible new controller services to the applicable API version.
          • Showing the implemented API versions for Controller Services.
          • Updating the property descriptor tooltip to indicate the required service requirements.
          • Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog.
          • Addressing remainder of the issues from recent rebase.

          Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting

          • Refactoring the way missing bundles work to retain the desired bundle if available
          • Fixing logger.isDebugEnabled to be logger.isTraceEnabled
          • Auditing when user changes the bundle. - Ensuring bundle details are present in templates.

          Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory

          • Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone.

          Refactoring controller service invocation handler to allow updating the node used by the invocation handler

          • Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow
          • Merging whether multiple versions of the component are available
          • Setting NAR plugin back to current released version
          • Cleaning up DocGenerator to not process multiple times

          Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs.

          Improving error messages when looking for bundles

          Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests.

          Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type.

          Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0

          Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster

          NIFI-3380: NIFI-3520: - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied.

          Fixing checkstyles issue and adding back assume checks to distributed cache server test

          Ensuring new component types are sorted correctly when shown initially.

          This closes #1585.

          Show
          jira-bot ASF subversion and git services added a comment - Commit d90cf846b9dcd49acdeac7bb5fcd13ee80dcdba6 in nifi's branch refs/heads/master from Bryan Bende [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d90cf84 ] NIFI-3380 Bumping NAR plugin to 1.2.0-SNAPSHOT development to leverage changes from master, adding buildnumber-maven-plugin to nifi-nar-bundles to properly set build info in MANIFEST of NARs Refactoring NarDetails to include all info from MANIFEST Adding the concept of a Bundle and refactoring NarClassLoaders to pass Bundles to ExtensionManager Adding logic to fail start-up when multiple NARs with same coordinates exist, moving Bundle classes to framework API Refactoring bundle API to classes and creating BundleCoordinate Updating FlowController to use BundleCoordinate Updating the UI and DTO model to support showing bundle details that loaded an extension type. Adding bundle details for processor canvas node, processor dialogs, controller service dialogs, and reporting task dialogs. Updating the formating of the bundle coordinates. Addressing text overflow in the configuration/details dialog. Fixing self referencing functions. Updating extension UI mapping to incorporate bundle coordinates. Discovering custom UIs through the supplied bundles. Adding verification methods for creating extensions through the rest api. Only returning extensions that are common amongst all nodes. Rendering the ghost processors using a dotted border. Adding bundle details to the flow.xml. Loading NiFi build and version details from the framework NAR. Removing properties for build and version details. Wiring together front end and back end changes. Including bundle coordinates in the component data model. Wiring together component data model and flow.xml. Addressing issue when resolve unvesioned dependent NARs. Updating unit tests to pass based on framework changes Fixing logging of extension types during start up Allowing the application to start if there is a compatible bundle found. - Reporting missing bundle when the a compatible bundle is not found. - Fixing table height in new component dialogs. Fixing chechstyle error and increasing test timeout for TestStandardControllerServiceProvider Adding ability to change processor type at runtime Adding backend code to change type for controller services Cleaning up instance classloaders for temp components. Creating a dialog for changing the version of a component. Updating the formatting of the component type and bundle throughout. Updating the new component dialogs to support selecting source group. Cleaning up new component dialogs. Cleaning up documentation in the cluster node endpoint. Adding missing include in nifi-web-ui pom compressor plugin Refactoring so ConfigurableComponent provides getLogger() and so the nodes provide the ConfigurableComponent Creating LoggableComponent to pass around the component, logger, and coordinate with in the framework Finishing clean up following rebase. Calling lifecycle methods for add and remove when changing versions of a component Introducing verifyCanUpdateBundle(coordinate) to ConfiguredComponent, and adding unit tests Ensuring documentation is available for all components. Including those of the same type that are loaded from different bundles. Adding lookup from ClassLoader to Bundle, adding fix for instance class loading to include all parent NARs, and adding additional unit tests for FlowController Adding validation to ensure referenced controller services implement the required API Fixing template instantiation to look up compatible bundle Requiring services/reporting tasks to be disabled/stopped. Only supporting a change version option when the item has multiple versions available. Limiting the possible new controller services to the applicable API version. Showing the implemented API versions for Controller Services. Updating the property descriptor tooltip to indicate the required service requirements. Introducing version based sorting in the new component dialog, change version dialog, and new controller service dialog. Addressing remainder of the issues from recent rebase. Ensuring bundles have been added to the flow before proposing a flow, and incorporating bundle information into flow fingerprinting Refactoring the way missing bundles work to retain the desired bundle if available Fixing logger.isDebugEnabled to be logger.isTraceEnabled Auditing when user changes the bundle. - Ensuring bundle details are present in templates. Moving standard prioritizers to framework NAR and refactoring ExtensionManager logic to handle cases where an extension is in a JAR directly in the lib directory Ensuring all nodes attempt to instantiate the same template instance when the available bundles may differ. - Fixing the auditing of copy/paste and template instantiation. - Running addtional verification methods when running standalone. Refactoring controller service invocation handler to allow updating the node used by the invocation handler Ensuring the bundles in a proposed flow are compatible with the current instance when the current instance has no flow is going to accept the proposed flow Merging whether multiple versions of the component are available Setting NAR plugin back to current released version Cleaning up DocGenerator to not process multiple times Addressing incorrect usage of nf.Common. - Using formatType in the new component type dialogs. Improving error messages when looking for bundles Addressing comments from PR. - Fixing references to global nf namespace. - Fixing injection of nfProcessGroupConfiguration in nfComponentVersion. - Fixing web api integration tests. Not rendering unversioned in help documentation. - Ensuring the isExtentionMissing flag is correct after changing the component type. Adding synchronization in node classes to ensure changing component can't occur when component is running, introducing MissingBundleException for better reporting when a node can't join cluster due to a missing bundle, and bumping NAR plugin to released version 1.2.0 Adding concept of missing components to fingerprinting to ensure nodes agree on missing components when joining a cluster NIFI-3380 : NIFI-3520 : - Fixing hive nar dependency. - Marking DBCPService as provided. - Skipping services that require instance classloading and are cobundled with their service API. - Skipping components that require instance classloading and reference service APIs that are cobundled. - Addressing UI issues in the new component dialogs when re-opening with a filter applied. Fixing checkstyles issue and adding back assume checks to distributed cache server test Ensuring new component types are sorted correctly when shown initially. This closes #1585.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/1585

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1585
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user markap14 commented on the issue:

          https://github.com/apache/nifi/pull/1585

          This all looks good. I've run through the standard cases of standalone and clustered instances. Ran through some crazy cases that I could think of. Didn't run into any unexpected problems. Code looks good. I especially paid attention to the *Node classes, StandardFlowService, and Fingerprint classes when reviewing code but found nothing concerning. +1 merged to master!

          Show
          githubbot ASF GitHub Bot added a comment - Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/1585 This all looks good. I've run through the standard cases of standalone and clustered instances. Ran through some crazy cases that I could think of. Didn't run into any unexpected problems. Code looks good. I especially paid attention to the *Node classes, StandardFlowService, and Fingerprint classes when reviewing code but found nothing concerning. +1 merged to master!
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f0a1c14f025f6013ed3667ea4470cd34846756d7 in nifi's branch refs/heads/master from Bryan Bende
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f0a1c14 ]

          NIFI-3380 Fixing parent pom of nifi-standard-prioritizers to point to correct parent after moving under nifi-framework

          Show
          jira-bot ASF subversion and git services added a comment - Commit f0a1c14f025f6013ed3667ea4470cd34846756d7 in nifi's branch refs/heads/master from Bryan Bende [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f0a1c14 ] NIFI-3380 Fixing parent pom of nifi-standard-prioritizers to point to correct parent after moving under nifi-framework
          Hide
          mcgilman Matt Gilman added a comment -

          Reopening to address path issues in generated documentation.

          Show
          mcgilman Matt Gilman added a comment - Reopening to address path issues in generated documentation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user mcgilman opened a pull request:

          https://github.com/apache/nifi/pull/1627

          NIFI-3380: Fixing paths in generated documentation

          NIFI-3380:

          • Addressing issues with paths in generated documentation.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/mcgilman/nifi NIFI-3380

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/1627.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1627


          commit b03b9b70fb4529498cd6d1717afea25a19d8c73f
          Author: Matt Gilman <matt.c.gilman@gmail.com>
          Date: 2017-03-27T14:21:01Z

          NIFI-3380:

          • Addressing issues with paths in generated documentation.

          Show
          githubbot ASF GitHub Bot added a comment - GitHub user mcgilman opened a pull request: https://github.com/apache/nifi/pull/1627 NIFI-3380 : Fixing paths in generated documentation NIFI-3380 : Addressing issues with paths in generated documentation. You can merge this pull request into a Git repository by running: $ git pull https://github.com/mcgilman/nifi NIFI-3380 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1627.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1627 commit b03b9b70fb4529498cd6d1717afea25a19d8c73f Author: Matt Gilman <matt.c.gilman@gmail.com> Date: 2017-03-27T14:21:01Z NIFI-3380 : Addressing issues with paths in generated documentation.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user bbende commented on the issue:

          https://github.com/apache/nifi/pull/1627

          Looks good, will merge to master

          Show
          githubbot ASF GitHub Bot added a comment - Github user bbende commented on the issue: https://github.com/apache/nifi/pull/1627 Looks good, will merge to master
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit f15e0d8e1e5b7cbb727b2a6604d807c089e8abed in nifi's branch refs/heads/master from Matt Gilman
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f15e0d8 ]

          NIFI-3380: - Addressing issues with paths in generated documentation.

          This closes #1627.

          Signed-off-by: Bryan Bende <bbende@apache.org>

          Show
          jira-bot ASF subversion and git services added a comment - Commit f15e0d8e1e5b7cbb727b2a6604d807c089e8abed in nifi's branch refs/heads/master from Matt Gilman [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=f15e0d8 ] NIFI-3380 : - Addressing issues with paths in generated documentation. This closes #1627. Signed-off-by: Bryan Bende <bbende@apache.org>
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/1627

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1627

            People

            • Assignee:
              mcgilman Matt Gilman
              Reporter:
              bende Bryan Bende
            • Votes:
              1 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development