Details

      Description

      Now that NIFI-210 adds scriptable Processors (and scripted onTrigger bodies), a great extension would be to add a scriptable ReportingTask. This would enable users to script their own ReportingTasks using the various supported scripting languages.

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user mattyb149 opened a pull request:

          https://github.com/apache/nifi/pull/1045

          NIFI-1458: Added ScriptedReportingTask

          I refactored AbstractScriptProcessor into ScriptUtils, so the functionality could be shared between scripting processors and scripting reporting tasks. I admit the use of ScriptUtils is not pure OO (public members, e.g.) but I wanted to keep the spirit of what the processors did when this was part of its parent class (since we don't have traits or multiple inheritance).

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/mattyb149/nifi NIFI-1458

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/1045.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #1045


          commit cd5a523b41cbe81a602e4d0f10df38b6a58a6779
          Author: Matt Burgess <mattyb149@apache.org>
          Date: 2016-09-22T14:13:40Z

          NIFI-1458: Added ScriptedReportingTask


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/1045 NIFI-1458 : Added ScriptedReportingTask I refactored AbstractScriptProcessor into ScriptUtils, so the functionality could be shared between scripting processors and scripting reporting tasks. I admit the use of ScriptUtils is not pure OO (public members, e.g.) but I wanted to keep the spirit of what the processors did when this was part of its parent class (since we don't have traits or multiple inheritance). You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-1458 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1045.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1045 commit cd5a523b41cbe81a602e4d0f10df38b6a58a6779 Author: Matt Burgess <mattyb149@apache.org> Date: 2016-09-22T14:13:40Z NIFI-1458 : Added ScriptedReportingTask
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on the issue:

          https://github.com/apache/nifi/pull/1045

          Reviewing...

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1045 Reviewing...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r83518147

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java —
          @@ -66,8 +85,8 @@
          @Override
          public Set<Relationship> getRelationships() {
          final Set<Relationship> relationships = new HashSet<>();

          • relationships.add(REL_SUCCESS);
          • relationships.add(REL_FAILURE);
            + relationships.add(ScriptUtils.REL_SUCCESS);
              • End diff –

          Can these two lines just reference the local `REL_SUCCESS` and `REL_FAILURE` rather than the external static variables?

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r83518147 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java — @@ -66,8 +85,8 @@ @Override public Set<Relationship> getRelationships() { final Set<Relationship> relationships = new HashSet<>(); relationships.add(REL_SUCCESS); relationships.add(REL_FAILURE); + relationships.add(ScriptUtils.REL_SUCCESS); End diff – Can these two lines just reference the local `REL_SUCCESS` and `REL_FAILURE` rather than the external static variables?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r83522085

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java —
          @@ -177,8 +199,8 @@ public void onTrigger(ProcessContext context, ProcessSessionFactory sessionFacto
          bindings.put("session", session);
          bindings.put("context", context);
          bindings.put("log", log);

          • bindings.put("REL_SUCCESS", REL_SUCCESS);
          • bindings.put("REL_FAILURE", REL_FAILURE);
            + bindings.put("REL_SUCCESS", ScriptUtils.REL_SUCCESS);
              • End diff –

          Same comment as above.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r83522085 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java — @@ -177,8 +199,8 @@ public void onTrigger(ProcessContext context, ProcessSessionFactory sessionFacto bindings.put("session", session); bindings.put("context", context); bindings.put("log", log); bindings.put("REL_SUCCESS", REL_SUCCESS); bindings.put("REL_FAILURE", REL_FAILURE); + bindings.put("REL_SUCCESS", ScriptUtils.REL_SUCCESS); End diff – Same comment as above.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r83524685

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java —
          @@ -46,23 +44,19 @@
          import javax.script.ScriptEngineManager;
          import javax.script.ScriptException;

          -import org.apache.nifi.annotation.lifecycle.OnStopped;
          import org.apache.nifi.components.AllowableValue;
          import org.apache.nifi.components.PropertyDescriptor;
          import org.apache.nifi.components.ValidationContext;
          import org.apache.nifi.components.ValidationResult;
          import org.apache.nifi.components.Validator;
          -import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
          import org.apache.nifi.processor.Relationship;
          import org.apache.nifi.processor.util.StandardValidators;
          import org.apache.nifi.util.StringUtils;

          /**

          • * This class contains variables and methods common to scripting processors
            + * This class contains variables and methods common to scripting processors, reporting tasks, etc.
            */
            -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER}

            ,

          • description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.")
            -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor {
            +public class ScriptUtils {
              • End diff –

          I'm now realizing this is because it was previously the parent class for the processors. As much as it pains me to ask, I think the mental benefit of separating state container logic from the util methods & static constants outweighs the complexity of introducing a second class. Interested to hear from other people though; this may just be a personal preference.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r83524685 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java — @@ -46,23 +44,19 @@ import javax.script.ScriptEngineManager; import javax.script.ScriptException; -import org.apache.nifi.annotation.lifecycle.OnStopped; import org.apache.nifi.components.AllowableValue; import org.apache.nifi.components.PropertyDescriptor; import org.apache.nifi.components.ValidationContext; import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.Validator; -import org.apache.nifi.processor.AbstractSessionFactoryProcessor; import org.apache.nifi.processor.Relationship; import org.apache.nifi.processor.util.StandardValidators; import org.apache.nifi.util.StringUtils; /** * This class contains variables and methods common to scripting processors + * This class contains variables and methods common to scripting processors, reporting tasks, etc. */ -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER} , description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.") -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor { +public class ScriptUtils { End diff – I'm now realizing this is because it was previously the parent class for the processors. As much as it pains me to ask, I think the mental benefit of separating state container logic from the util methods & static constants outweighs the complexity of introducing a second class. Interested to hear from other people though; this may just be a personal preference.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r83522194

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java —
          @@ -46,23 +44,19 @@
          import javax.script.ScriptEngineManager;
          import javax.script.ScriptException;

          -import org.apache.nifi.annotation.lifecycle.OnStopped;
          import org.apache.nifi.components.AllowableValue;
          import org.apache.nifi.components.PropertyDescriptor;
          import org.apache.nifi.components.ValidationContext;
          import org.apache.nifi.components.ValidationResult;
          import org.apache.nifi.components.Validator;
          -import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
          import org.apache.nifi.processor.Relationship;
          import org.apache.nifi.processor.util.StandardValidators;
          import org.apache.nifi.util.StringUtils;

          /**

          • * This class contains variables and methods common to scripting processors
            + * This class contains variables and methods common to scripting processors, reporting tasks, etc.
            */
            -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER}

            ,

          • description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.")
            -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor {
            +public class ScriptUtils {
              • End diff –

          I'm a little confused – it seems like this class both contains static constants and utility methods (which I would expect to be together) as well acting as a state container for dynamic values necessary for the `ExecuteScript` or `InvokeScriptedProcessor` components. Am I missing something?

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r83522194 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java — @@ -46,23 +44,19 @@ import javax.script.ScriptEngineManager; import javax.script.ScriptException; -import org.apache.nifi.annotation.lifecycle.OnStopped; import org.apache.nifi.components.AllowableValue; import org.apache.nifi.components.PropertyDescriptor; import org.apache.nifi.components.ValidationContext; import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.Validator; -import org.apache.nifi.processor.AbstractSessionFactoryProcessor; import org.apache.nifi.processor.Relationship; import org.apache.nifi.processor.util.StandardValidators; import org.apache.nifi.util.StringUtils; /** * This class contains variables and methods common to scripting processors + * This class contains variables and methods common to scripting processors, reporting tasks, etc. */ -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER} , description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.") -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor { +public class ScriptUtils { End diff – I'm a little confused – it seems like this class both contains static constants and utility methods (which I would expect to be together) as well acting as a state container for dynamic values necessary for the `ExecuteScript` or `InvokeScriptedProcessor` components. Am I missing something?
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r83522047

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java —
          @@ -116,30 +139,29 @@ protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String
          */
          @OnScheduled
          public void setup(final ProcessContext context) {

          • scriptEngineName = context.getProperty(SCRIPT_ENGINE).getValue();
          • scriptPath = context.getProperty(SCRIPT_FILE).evaluateAttributeExpressions().getValue();
          • scriptBody = context.getProperty(SCRIPT_BODY).getValue();
          • String modulePath = context.getProperty(MODULES).getValue();
            + scriptUtils.scriptEngineName = context.getProperty(ScriptUtils.SCRIPT_ENGINE).getValue();
              • End diff –

          Could this initialization process be encapsulated in a method (either here or within `ScriptUtils`) that accepts the `context` as a parameter? Seems like a minor logic leak to iterate over the constants here.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r83522047 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java — @@ -116,30 +139,29 @@ protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String */ @OnScheduled public void setup(final ProcessContext context) { scriptEngineName = context.getProperty(SCRIPT_ENGINE).getValue(); scriptPath = context.getProperty(SCRIPT_FILE).evaluateAttributeExpressions().getValue(); scriptBody = context.getProperty(SCRIPT_BODY).getValue(); String modulePath = context.getProperty(MODULES).getValue(); + scriptUtils.scriptEngineName = context.getProperty(ScriptUtils.SCRIPT_ENGINE).getValue(); End diff – Could this initialization process be encapsulated in a method (either here or within `ScriptUtils`) that accepts the `context` as a parameter? Seems like a minor logic leak to iterate over the constants here.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r83525123

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/reporting/script/ScriptedReportingTask.java —
          @@ -0,0 +1,197 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.nifi.reporting.script;
          +
          +import com.yammer.metrics.core.VirtualMachineMetrics;
          +import org.apache.commons.io.IOUtils;
          +import org.apache.nifi.annotation.documentation.CapabilityDescription;
          +import org.apache.nifi.annotation.documentation.Tags;
          +import org.apache.nifi.annotation.lifecycle.OnScheduled;
          +import org.apache.nifi.components.PropertyDescriptor;
          +import org.apache.nifi.components.ValidationContext;
          +import org.apache.nifi.components.ValidationResult;
          +import org.apache.nifi.controller.ConfigurationContext;
          +import org.apache.nifi.logging.ComponentLog;
          +import org.apache.nifi.processor.exception.ProcessException;
          +import org.apache.nifi.processor.util.StandardValidators;
          +import org.apache.nifi.processors.script.ScriptEngineConfigurator;
          +import org.apache.nifi.processors.script.ScriptUtils;
          +import org.apache.nifi.reporting.AbstractReportingTask;
          +import org.apache.nifi.reporting.ReportingContext;
          +import org.apache.nifi.util.StringUtils;
          +
          +import javax.script.Bindings;
          +import javax.script.ScriptContext;
          +import javax.script.ScriptEngine;
          +import javax.script.ScriptException;
          +import javax.script.SimpleBindings;
          +import java.io.FileInputStream;
          +import java.io.IOException;
          +import java.nio.charset.Charset;
          +import java.util.Collection;
          +import java.util.Collections;
          +import java.util.List;
          +import java.util.Map;
          +
          +/**
          + * A Reporting task whose body is provided by a script (via supported JSR-223 script engines)
          + */
          +@Tags(

          {"reporting", "script", "execute", "groovy", "python", "jython", "jruby", "ruby", "javascript", "js", "lua", "luaj"}

          )
          +@CapabilityDescription("Provides reporting and status information to a script. ReportingContext, ComponentLog, and VirtualMachineMetrics objects are made available "
          + + "as variables (context, log, and vmMetrics, respectively) to the script for further processing. The context makes various information available such "
          + + "as events, provenance, bulletins, controller services, process groups, Java Virtual Machine metrics, etc.")
          +public class ScriptedReportingTask extends AbstractReportingTask {
          +
          + protected volatile ScriptUtils scriptUtils = new ScriptUtils();
          + protected volatile String scriptToRun = null;
          + protected volatile VirtualMachineMetrics vmMetrics;
          +
          + /**
          + * Returns a list of property descriptors supported by this processor. The list always includes properties such as
          + * script engine name, script file name, script body name, script arguments, and an external module path. If the
          + * scripted processor also defines supported properties, those are added to the list as well.
          + *
          + * @return a List of PropertyDescriptor objects supported by this processor
          + */
          + @Override
          + protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
          + synchronized (scriptUtils.isInitialized) {
          + if (!scriptUtils.isInitialized.get())

          { + scriptUtils.createResources(); + }

          + }
          +
          + return Collections.unmodifiableList(scriptUtils.descriptors);
          + }
          +
          + /**
          + * Returns a PropertyDescriptor for the given name. This is for the user to be able to define their own properties
          + * which will be available as variables in the script
          + *
          + * @param propertyDescriptorName used to lookup if any property descriptors exist for that name
          + * @return a PropertyDescriptor object corresponding to the specified dynamic property name
          + */
          + @Override
          + protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String propertyDescriptorName) {
          + return new PropertyDescriptor.Builder()
          + .name(propertyDescriptorName)
          — End diff –

          Obviously the name is provided by the user at runtime, but should we still try to define a `displayName` for the `PropertyDescriptor`? I could see three possibilities:

          • `name` and `displayName` are identical – not much value
          • `name` is as provided, `displayName` is the parameter passed through a `humanize()`-type method (see below)
          • `displayName` is as provided, `name` is the parameter passed through a `symbolize()`-type method (see below)

          The transform methods `humanize()`/`symbolize()` could be as simple as title-casing and adding/removing spaces at word boundary guesses.

          I'm not sure how much energy this is worth investing, as these property names are not constants which will be retained across instances of the processor, but I think they still would lose the state persisted in the `flow.xml.gz` if it was somehow modified.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r83525123 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/reporting/script/ScriptedReportingTask.java — @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.reporting.script; + +import com.yammer.metrics.core.VirtualMachineMetrics; +import org.apache.commons.io.IOUtils; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.script.ScriptEngineConfigurator; +import org.apache.nifi.processors.script.ScriptUtils; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.util.StringUtils; + +import javax.script.Bindings; +import javax.script.ScriptContext; +import javax.script.ScriptEngine; +import javax.script.ScriptException; +import javax.script.SimpleBindings; +import java.io.FileInputStream; +import java.io.IOException; +import java.nio.charset.Charset; +import java.util.Collection; +import java.util.Collections; +import java.util.List; +import java.util.Map; + +/** + * A Reporting task whose body is provided by a script (via supported JSR-223 script engines) + */ +@Tags( {"reporting", "script", "execute", "groovy", "python", "jython", "jruby", "ruby", "javascript", "js", "lua", "luaj"} ) +@CapabilityDescription("Provides reporting and status information to a script. ReportingContext, ComponentLog, and VirtualMachineMetrics objects are made available " + + "as variables (context, log, and vmMetrics, respectively) to the script for further processing. The context makes various information available such " + + "as events, provenance, bulletins, controller services, process groups, Java Virtual Machine metrics, etc.") +public class ScriptedReportingTask extends AbstractReportingTask { + + protected volatile ScriptUtils scriptUtils = new ScriptUtils(); + protected volatile String scriptToRun = null; + protected volatile VirtualMachineMetrics vmMetrics; + + /** + * Returns a list of property descriptors supported by this processor. The list always includes properties such as + * script engine name, script file name, script body name, script arguments, and an external module path. If the + * scripted processor also defines supported properties, those are added to the list as well. + * + * @return a List of PropertyDescriptor objects supported by this processor + */ + @Override + protected List<PropertyDescriptor> getSupportedPropertyDescriptors() { + synchronized (scriptUtils.isInitialized) { + if (!scriptUtils.isInitialized.get()) { + scriptUtils.createResources(); + } + } + + return Collections.unmodifiableList(scriptUtils.descriptors); + } + + /** + * Returns a PropertyDescriptor for the given name. This is for the user to be able to define their own properties + * which will be available as variables in the script + * + * @param propertyDescriptorName used to lookup if any property descriptors exist for that name + * @return a PropertyDescriptor object corresponding to the specified dynamic property name + */ + @Override + protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String propertyDescriptorName) { + return new PropertyDescriptor.Builder() + .name(propertyDescriptorName) — End diff – Obviously the name is provided by the user at runtime, but should we still try to define a `displayName` for the `PropertyDescriptor`? I could see three possibilities: `name` and `displayName` are identical – not much value `name` is as provided, `displayName` is the parameter passed through a `humanize()`-type method (see below) `displayName` is as provided, `name` is the parameter passed through a `symbolize()`-type method (see below) The transform methods `humanize()`/`symbolize()` could be as simple as title-casing and adding/removing spaces at word boundary guesses. I'm not sure how much energy this is worth investing, as these property names are not constants which will be retained across instances of the processor, but I think they still would lose the state persisted in the `flow.xml.gz` if it was somehow modified.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r83524883

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java —
          @@ -46,23 +44,19 @@
          import javax.script.ScriptEngineManager;
          import javax.script.ScriptException;

          -import org.apache.nifi.annotation.lifecycle.OnStopped;
          import org.apache.nifi.components.AllowableValue;
          import org.apache.nifi.components.PropertyDescriptor;
          import org.apache.nifi.components.ValidationContext;
          import org.apache.nifi.components.ValidationResult;
          import org.apache.nifi.components.Validator;
          -import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
          import org.apache.nifi.processor.Relationship;
          import org.apache.nifi.processor.util.StandardValidators;
          import org.apache.nifi.util.StringUtils;

          /**

          • * This class contains variables and methods common to scripting processors
            + * This class contains variables and methods common to scripting processors, reporting tasks, etc.
            */
            -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER}

            ,

          • description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.")
            -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor {
            +public class ScriptUtils {
              • End diff –

          Sorry to spam; I think splitting them would also help mitigate your concerns about exposing public members, as they would be fields of a POJO with accessors.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r83524883 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java — @@ -46,23 +44,19 @@ import javax.script.ScriptEngineManager; import javax.script.ScriptException; -import org.apache.nifi.annotation.lifecycle.OnStopped; import org.apache.nifi.components.AllowableValue; import org.apache.nifi.components.PropertyDescriptor; import org.apache.nifi.components.ValidationContext; import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.Validator; -import org.apache.nifi.processor.AbstractSessionFactoryProcessor; import org.apache.nifi.processor.Relationship; import org.apache.nifi.processor.util.StandardValidators; import org.apache.nifi.util.StringUtils; /** * This class contains variables and methods common to scripting processors + * This class contains variables and methods common to scripting processors, reporting tasks, etc. */ -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER} , description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.") -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor { +public class ScriptUtils { End diff – Sorry to spam; I think splitting them would also help mitigate your concerns about exposing public members, as they would be fields of a POJO with accessors.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r85116446

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java —
          @@ -66,8 +85,8 @@
          @Override
          public Set<Relationship> getRelationships() {
          final Set<Relationship> relationships = new HashSet<>();

          • relationships.add(REL_SUCCESS);
          • relationships.add(REL_FAILURE);
            + relationships.add(ScriptUtils.REL_SUCCESS);
              • End diff –

          Yes, I only changed them because I kept the class-level versions of them for backwards compatibility

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r85116446 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java — @@ -66,8 +85,8 @@ @Override public Set<Relationship> getRelationships() { final Set<Relationship> relationships = new HashSet<>(); relationships.add(REL_SUCCESS); relationships.add(REL_FAILURE); + relationships.add(ScriptUtils.REL_SUCCESS); End diff – Yes, I only changed them because I kept the class-level versions of them for backwards compatibility
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r85117619

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java —
          @@ -116,30 +139,29 @@ protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String
          */
          @OnScheduled
          public void setup(final ProcessContext context) {

          • scriptEngineName = context.getProperty(SCRIPT_ENGINE).getValue();
          • scriptPath = context.getProperty(SCRIPT_FILE).evaluateAttributeExpressions().getValue();
          • scriptBody = context.getProperty(SCRIPT_BODY).getValue();
          • String modulePath = context.getProperty(MODULES).getValue();
            + scriptUtils.scriptEngineName = context.getProperty(ScriptUtils.SCRIPT_ENGINE).getValue();
              • End diff –

          Will do

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r85117619 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ExecuteScript.java — @@ -116,30 +139,29 @@ protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String */ @OnScheduled public void setup(final ProcessContext context) { scriptEngineName = context.getProperty(SCRIPT_ENGINE).getValue(); scriptPath = context.getProperty(SCRIPT_FILE).evaluateAttributeExpressions().getValue(); scriptBody = context.getProperty(SCRIPT_BODY).getValue(); String modulePath = context.getProperty(MODULES).getValue(); + scriptUtils.scriptEngineName = context.getProperty(ScriptUtils.SCRIPT_ENGINE).getValue(); End diff – Will do
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r85119222

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java —
          @@ -46,23 +44,19 @@
          import javax.script.ScriptEngineManager;
          import javax.script.ScriptException;

          -import org.apache.nifi.annotation.lifecycle.OnStopped;
          import org.apache.nifi.components.AllowableValue;
          import org.apache.nifi.components.PropertyDescriptor;
          import org.apache.nifi.components.ValidationContext;
          import org.apache.nifi.components.ValidationResult;
          import org.apache.nifi.components.Validator;
          -import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
          import org.apache.nifi.processor.Relationship;
          import org.apache.nifi.processor.util.StandardValidators;
          import org.apache.nifi.util.StringUtils;

          /**

          • * This class contains variables and methods common to scripting processors
            + * This class contains variables and methods common to scripting processors, reporting tasks, etc.
            */
            -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER}

            ,

          • description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.")
            -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor {
            +public class ScriptUtils {
              • End diff –

          How do you picture the split? If I move the PropertyDescriptors and Relationships out, then I possibly should update all the state container logic such that those are passed in vs. used from the static constants class. I took quite a few liberties with the OO on this because it would involve a large amount of refactor and I didn't want to risk introducing more bugs. If you've got a good proposal for this though, I'd be happy to make the change

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r85119222 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java — @@ -46,23 +44,19 @@ import javax.script.ScriptEngineManager; import javax.script.ScriptException; -import org.apache.nifi.annotation.lifecycle.OnStopped; import org.apache.nifi.components.AllowableValue; import org.apache.nifi.components.PropertyDescriptor; import org.apache.nifi.components.ValidationContext; import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.Validator; -import org.apache.nifi.processor.AbstractSessionFactoryProcessor; import org.apache.nifi.processor.Relationship; import org.apache.nifi.processor.util.StandardValidators; import org.apache.nifi.util.StringUtils; /** * This class contains variables and methods common to scripting processors + * This class contains variables and methods common to scripting processors, reporting tasks, etc. */ -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER} , description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.") -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor { +public class ScriptUtils { End diff – How do you picture the split? If I move the PropertyDescriptors and Relationships out, then I possibly should update all the state container logic such that those are passed in vs. used from the static constants class. I took quite a few liberties with the OO on this because it would involve a large amount of refactor and I didn't want to risk introducing more bugs. If you've got a good proposal for this though, I'd be happy to make the change
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r85119526

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/reporting/script/ScriptedReportingTask.java —
          @@ -0,0 +1,197 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.nifi.reporting.script;
          +
          +import com.yammer.metrics.core.VirtualMachineMetrics;
          +import org.apache.commons.io.IOUtils;
          +import org.apache.nifi.annotation.documentation.CapabilityDescription;
          +import org.apache.nifi.annotation.documentation.Tags;
          +import org.apache.nifi.annotation.lifecycle.OnScheduled;
          +import org.apache.nifi.components.PropertyDescriptor;
          +import org.apache.nifi.components.ValidationContext;
          +import org.apache.nifi.components.ValidationResult;
          +import org.apache.nifi.controller.ConfigurationContext;
          +import org.apache.nifi.logging.ComponentLog;
          +import org.apache.nifi.processor.exception.ProcessException;
          +import org.apache.nifi.processor.util.StandardValidators;
          +import org.apache.nifi.processors.script.ScriptEngineConfigurator;
          +import org.apache.nifi.processors.script.ScriptUtils;
          +import org.apache.nifi.reporting.AbstractReportingTask;
          +import org.apache.nifi.reporting.ReportingContext;
          +import org.apache.nifi.util.StringUtils;
          +
          +import javax.script.Bindings;
          +import javax.script.ScriptContext;
          +import javax.script.ScriptEngine;
          +import javax.script.ScriptException;
          +import javax.script.SimpleBindings;
          +import java.io.FileInputStream;
          +import java.io.IOException;
          +import java.nio.charset.Charset;
          +import java.util.Collection;
          +import java.util.Collections;
          +import java.util.List;
          +import java.util.Map;
          +
          +/**
          + * A Reporting task whose body is provided by a script (via supported JSR-223 script engines)
          + */
          +@Tags(

          {"reporting", "script", "execute", "groovy", "python", "jython", "jruby", "ruby", "javascript", "js", "lua", "luaj"}

          )
          +@CapabilityDescription("Provides reporting and status information to a script. ReportingContext, ComponentLog, and VirtualMachineMetrics objects are made available "
          + + "as variables (context, log, and vmMetrics, respectively) to the script for further processing. The context makes various information available such "
          + + "as events, provenance, bulletins, controller services, process groups, Java Virtual Machine metrics, etc.")
          +public class ScriptedReportingTask extends AbstractReportingTask {
          +
          + protected volatile ScriptUtils scriptUtils = new ScriptUtils();
          + protected volatile String scriptToRun = null;
          + protected volatile VirtualMachineMetrics vmMetrics;
          +
          + /**
          + * Returns a list of property descriptors supported by this processor. The list always includes properties such as
          + * script engine name, script file name, script body name, script arguments, and an external module path. If the
          + * scripted processor also defines supported properties, those are added to the list as well.
          + *
          + * @return a List of PropertyDescriptor objects supported by this processor
          + */
          + @Override
          + protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
          + synchronized (scriptUtils.isInitialized) {
          + if (!scriptUtils.isInitialized.get())

          { + scriptUtils.createResources(); + }

          + }
          +
          + return Collections.unmodifiableList(scriptUtils.descriptors);
          + }
          +
          + /**
          + * Returns a PropertyDescriptor for the given name. This is for the user to be able to define their own properties
          + * which will be available as variables in the script
          + *
          + * @param propertyDescriptorName used to lookup if any property descriptors exist for that name
          + * @return a PropertyDescriptor object corresponding to the specified dynamic property name
          + */
          + @Override
          + protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String propertyDescriptorName) {
          + return new PropertyDescriptor.Builder()
          + .name(propertyDescriptorName)
          — End diff –

          This is a good point and applies everywhere for dynamic properties, I think we should file a Jira to investigate.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r85119526 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/reporting/script/ScriptedReportingTask.java — @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.reporting.script; + +import com.yammer.metrics.core.VirtualMachineMetrics; +import org.apache.commons.io.IOUtils; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.script.ScriptEngineConfigurator; +import org.apache.nifi.processors.script.ScriptUtils; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.util.StringUtils; + +import javax.script.Bindings; +import javax.script.ScriptContext; +import javax.script.ScriptEngine; +import javax.script.ScriptException; +import javax.script.SimpleBindings; +import java.io.FileInputStream; +import java.io.IOException; +import java.nio.charset.Charset; +import java.util.Collection; +import java.util.Collections; +import java.util.List; +import java.util.Map; + +/** + * A Reporting task whose body is provided by a script (via supported JSR-223 script engines) + */ +@Tags( {"reporting", "script", "execute", "groovy", "python", "jython", "jruby", "ruby", "javascript", "js", "lua", "luaj"} ) +@CapabilityDescription("Provides reporting and status information to a script. ReportingContext, ComponentLog, and VirtualMachineMetrics objects are made available " + + "as variables (context, log, and vmMetrics, respectively) to the script for further processing. The context makes various information available such " + + "as events, provenance, bulletins, controller services, process groups, Java Virtual Machine metrics, etc.") +public class ScriptedReportingTask extends AbstractReportingTask { + + protected volatile ScriptUtils scriptUtils = new ScriptUtils(); + protected volatile String scriptToRun = null; + protected volatile VirtualMachineMetrics vmMetrics; + + /** + * Returns a list of property descriptors supported by this processor. The list always includes properties such as + * script engine name, script file name, script body name, script arguments, and an external module path. If the + * scripted processor also defines supported properties, those are added to the list as well. + * + * @return a List of PropertyDescriptor objects supported by this processor + */ + @Override + protected List<PropertyDescriptor> getSupportedPropertyDescriptors() { + synchronized (scriptUtils.isInitialized) { + if (!scriptUtils.isInitialized.get()) { + scriptUtils.createResources(); + } + } + + return Collections.unmodifiableList(scriptUtils.descriptors); + } + + /** + * Returns a PropertyDescriptor for the given name. This is for the user to be able to define their own properties + * which will be available as variables in the script + * + * @param propertyDescriptorName used to lookup if any property descriptors exist for that name + * @return a PropertyDescriptor object corresponding to the specified dynamic property name + */ + @Override + protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String propertyDescriptorName) { + return new PropertyDescriptor.Builder() + .name(propertyDescriptorName) — End diff – This is a good point and applies everywhere for dynamic properties, I think we should file a Jira to investigate.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r85123335

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java —
          @@ -46,23 +44,19 @@
          import javax.script.ScriptEngineManager;
          import javax.script.ScriptException;

          -import org.apache.nifi.annotation.lifecycle.OnStopped;
          import org.apache.nifi.components.AllowableValue;
          import org.apache.nifi.components.PropertyDescriptor;
          import org.apache.nifi.components.ValidationContext;
          import org.apache.nifi.components.ValidationResult;
          import org.apache.nifi.components.Validator;
          -import org.apache.nifi.processor.AbstractSessionFactoryProcessor;
          import org.apache.nifi.processor.Relationship;
          import org.apache.nifi.processor.util.StandardValidators;
          import org.apache.nifi.util.StringUtils;

          /**

          • * This class contains variables and methods common to scripting processors
            + * This class contains variables and methods common to scripting processors, reporting tasks, etc.
            */
            -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER}

            ,

          • description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.")
            -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor {
            +public class ScriptUtils {
              • End diff –

          I can also change the public members to private and change the rest of the classes to use the accessors. I chose not to only to make the changes easier to see (and probably a bit of laziness)

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r85123335 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptUtils.java — @@ -46,23 +44,19 @@ import javax.script.ScriptEngineManager; import javax.script.ScriptException; -import org.apache.nifi.annotation.lifecycle.OnStopped; import org.apache.nifi.components.AllowableValue; import org.apache.nifi.components.PropertyDescriptor; import org.apache.nifi.components.ValidationContext; import org.apache.nifi.components.ValidationResult; import org.apache.nifi.components.Validator; -import org.apache.nifi.processor.AbstractSessionFactoryProcessor; import org.apache.nifi.processor.Relationship; import org.apache.nifi.processor.util.StandardValidators; import org.apache.nifi.util.StringUtils; /** * This class contains variables and methods common to scripting processors + * This class contains variables and methods common to scripting processors, reporting tasks, etc. */ -@Stateful(scopes = {Scope.LOCAL, Scope.CLUSTER} , description = "Scripts can store and retrieve state using the State Management APIs. Consult the State Manager section of the Developer's Guide for more details.") -public abstract class AbstractScriptProcessor extends AbstractSessionFactoryProcessor { +public class ScriptUtils { End diff – I can also change the public members to private and change the rest of the classes to use the accessors. I chose not to only to make the changes easier to see (and probably a bit of laziness)
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/1045#discussion_r87113621

          — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/reporting/script/ScriptedReportingTask.java —
          @@ -0,0 +1,197 @@
          +/*
          + * Licensed to the Apache Software Foundation (ASF) under one or more
          + * contributor license agreements. See the NOTICE file distributed with
          + * this work for additional information regarding copyright ownership.
          + * The ASF licenses this file to You under the Apache License, Version 2.0
          + * (the "License"); you may not use this file except in compliance with
          + * the License. You may obtain a copy of the License at
          + *
          + * http://www.apache.org/licenses/LICENSE-2.0
          + *
          + * Unless required by applicable law or agreed to in writing, software
          + * distributed under the License is distributed on an "AS IS" BASIS,
          + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
          + * See the License for the specific language governing permissions and
          + * limitations under the License.
          + */
          +package org.apache.nifi.reporting.script;
          +
          +import com.yammer.metrics.core.VirtualMachineMetrics;
          +import org.apache.commons.io.IOUtils;
          +import org.apache.nifi.annotation.documentation.CapabilityDescription;
          +import org.apache.nifi.annotation.documentation.Tags;
          +import org.apache.nifi.annotation.lifecycle.OnScheduled;
          +import org.apache.nifi.components.PropertyDescriptor;
          +import org.apache.nifi.components.ValidationContext;
          +import org.apache.nifi.components.ValidationResult;
          +import org.apache.nifi.controller.ConfigurationContext;
          +import org.apache.nifi.logging.ComponentLog;
          +import org.apache.nifi.processor.exception.ProcessException;
          +import org.apache.nifi.processor.util.StandardValidators;
          +import org.apache.nifi.processors.script.ScriptEngineConfigurator;
          +import org.apache.nifi.processors.script.ScriptUtils;
          +import org.apache.nifi.reporting.AbstractReportingTask;
          +import org.apache.nifi.reporting.ReportingContext;
          +import org.apache.nifi.util.StringUtils;
          +
          +import javax.script.Bindings;
          +import javax.script.ScriptContext;
          +import javax.script.ScriptEngine;
          +import javax.script.ScriptException;
          +import javax.script.SimpleBindings;
          +import java.io.FileInputStream;
          +import java.io.IOException;
          +import java.nio.charset.Charset;
          +import java.util.Collection;
          +import java.util.Collections;
          +import java.util.List;
          +import java.util.Map;
          +
          +/**
          + * A Reporting task whose body is provided by a script (via supported JSR-223 script engines)
          + */
          +@Tags(

          {"reporting", "script", "execute", "groovy", "python", "jython", "jruby", "ruby", "javascript", "js", "lua", "luaj"}

          )
          +@CapabilityDescription("Provides reporting and status information to a script. ReportingContext, ComponentLog, and VirtualMachineMetrics objects are made available "
          + + "as variables (context, log, and vmMetrics, respectively) to the script for further processing. The context makes various information available such "
          + + "as events, provenance, bulletins, controller services, process groups, Java Virtual Machine metrics, etc.")
          +public class ScriptedReportingTask extends AbstractReportingTask {
          +
          + protected volatile ScriptUtils scriptUtils = new ScriptUtils();
          + protected volatile String scriptToRun = null;
          + protected volatile VirtualMachineMetrics vmMetrics;
          +
          + /**
          + * Returns a list of property descriptors supported by this processor. The list always includes properties such as
          + * script engine name, script file name, script body name, script arguments, and an external module path. If the
          + * scripted processor also defines supported properties, those are added to the list as well.
          + *
          + * @return a List of PropertyDescriptor objects supported by this processor
          + */
          + @Override
          + protected List<PropertyDescriptor> getSupportedPropertyDescriptors() {
          + synchronized (scriptUtils.isInitialized) {
          + if (!scriptUtils.isInitialized.get())

          { + scriptUtils.createResources(); + }

          + }
          +
          + return Collections.unmodifiableList(scriptUtils.descriptors);
          + }
          +
          + /**
          + * Returns a PropertyDescriptor for the given name. This is for the user to be able to define their own properties
          + * which will be available as variables in the script
          + *
          + * @param propertyDescriptorName used to lookup if any property descriptors exist for that name
          + * @return a PropertyDescriptor object corresponding to the specified dynamic property name
          + */
          + @Override
          + protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String propertyDescriptorName) {
          + return new PropertyDescriptor.Builder()
          + .name(propertyDescriptorName)
          — End diff –

          I filed NIFI-3006(https://issues.apache.org/jira/browse/NIFI-3006) to capture this. We can ignore it in this case until that Jira is worked on.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on a diff in the pull request: https://github.com/apache/nifi/pull/1045#discussion_r87113621 — Diff: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/reporting/script/ScriptedReportingTask.java — @@ -0,0 +1,197 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ +package org.apache.nifi.reporting.script; + +import com.yammer.metrics.core.VirtualMachineMetrics; +import org.apache.commons.io.IOUtils; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.controller.ConfigurationContext; +import org.apache.nifi.logging.ComponentLog; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.processor.util.StandardValidators; +import org.apache.nifi.processors.script.ScriptEngineConfigurator; +import org.apache.nifi.processors.script.ScriptUtils; +import org.apache.nifi.reporting.AbstractReportingTask; +import org.apache.nifi.reporting.ReportingContext; +import org.apache.nifi.util.StringUtils; + +import javax.script.Bindings; +import javax.script.ScriptContext; +import javax.script.ScriptEngine; +import javax.script.ScriptException; +import javax.script.SimpleBindings; +import java.io.FileInputStream; +import java.io.IOException; +import java.nio.charset.Charset; +import java.util.Collection; +import java.util.Collections; +import java.util.List; +import java.util.Map; + +/** + * A Reporting task whose body is provided by a script (via supported JSR-223 script engines) + */ +@Tags( {"reporting", "script", "execute", "groovy", "python", "jython", "jruby", "ruby", "javascript", "js", "lua", "luaj"} ) +@CapabilityDescription("Provides reporting and status information to a script. ReportingContext, ComponentLog, and VirtualMachineMetrics objects are made available " + + "as variables (context, log, and vmMetrics, respectively) to the script for further processing. The context makes various information available such " + + "as events, provenance, bulletins, controller services, process groups, Java Virtual Machine metrics, etc.") +public class ScriptedReportingTask extends AbstractReportingTask { + + protected volatile ScriptUtils scriptUtils = new ScriptUtils(); + protected volatile String scriptToRun = null; + protected volatile VirtualMachineMetrics vmMetrics; + + /** + * Returns a list of property descriptors supported by this processor. The list always includes properties such as + * script engine name, script file name, script body name, script arguments, and an external module path. If the + * scripted processor also defines supported properties, those are added to the list as well. + * + * @return a List of PropertyDescriptor objects supported by this processor + */ + @Override + protected List<PropertyDescriptor> getSupportedPropertyDescriptors() { + synchronized (scriptUtils.isInitialized) { + if (!scriptUtils.isInitialized.get()) { + scriptUtils.createResources(); + } + } + + return Collections.unmodifiableList(scriptUtils.descriptors); + } + + /** + * Returns a PropertyDescriptor for the given name. This is for the user to be able to define their own properties + * which will be available as variables in the script + * + * @param propertyDescriptorName used to lookup if any property descriptors exist for that name + * @return a PropertyDescriptor object corresponding to the specified dynamic property name + */ + @Override + protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final String propertyDescriptorName) { + return new PropertyDescriptor.Builder() + .name(propertyDescriptorName) — End diff – I filed NIFI-3006 ( https://issues.apache.org/jira/browse/NIFI-3006 ) to capture this. We can ignore it in this case until that Jira is worked on.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on the issue:

          https://github.com/apache/nifi/pull/1045

          I think most everything was hashed out sufficiently. If you make the changes you indicated above, I'll run locally, verify `contrib-check`, and test, and provide the +1.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1045 I think most everything was hashed out sufficiently. If you make the changes you indicated above, I'll run locally, verify `contrib-check`, and test, and provide the +1.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user alopresto commented on the issue:

          https://github.com/apache/nifi/pull/1045

          @mattyb149 everything looks great; I tried it out and was able to write a simple Groovy script and run it as a reporting task. One thing is that since you submitted this, we've added the `Restricted` designator for "dangerous" processors/controller services/reporting tasks, and I feel this counts. If you add that, I'll merge this in. Thanks.

          Show
          githubbot ASF GitHub Bot added a comment - Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/1045 @mattyb149 everything looks great; I tried it out and was able to write a simple Groovy script and run it as a reporting task. One thing is that since you submitted this, we've added the `Restricted` designator for "dangerous" processors/controller services/reporting tasks, and I feel this counts. If you add that, I'll merge this in. Thanks.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on the issue:

          https://github.com/apache/nifi/pull/1045

          Absolutely will add Restricted, good catch thanks!

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1045 Absolutely will add Restricted, good catch thanks!
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mattyb149 commented on the issue:

          https://github.com/apache/nifi/pull/1045

          Also I noticed the DynamicProperty annotation is missing, will add that too.

          Show
          githubbot ASF GitHub Bot added a comment - Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1045 Also I noticed the DynamicProperty annotation is missing, will add that too.
          Hide
          jira-bot ASF subversion and git services added a comment -

          Commit 675f4f544cb87bef0ec51b8dee5023088ad224ad in nifi's branch refs/heads/master from Matt Burgess
          [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=675f4f5 ]

          NIFI-1458: Added ScriptedReportingTask

          This closes #1045.

          Signed-off-by: Andy LoPresto <alopresto@apache.org>

          Show
          jira-bot ASF subversion and git services added a comment - Commit 675f4f544cb87bef0ec51b8dee5023088ad224ad in nifi's branch refs/heads/master from Matt Burgess [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=675f4f5 ] NIFI-1458 : Added ScriptedReportingTask This closes #1045. Signed-off-by: Andy LoPresto <alopresto@apache.org>
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/1045

          Show
          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1045

            People

            • Assignee:
              mattyb149 Matt Burgess
              Reporter:
              mattyb149 Matt Burgess
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development