Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 1.8.0, 1.7.1
    • 1.8.0
    • Core Framework
    • None

    Description

      We are using a LookupRecord processor to add an array property to a Record.  The LookupRecord's `Lookup Service` is a MongoDBLookupService. The idea is for the LookupRecord to query Mongo documents by the Record's `_id`, to extract the array property at `f` on that document, and to insert that array into the Record at the RecordPath `f`. This flow results in the conversion error copied below, which completely prevents the FlowFile from proceeding (routes to Failure/Unmatched.) It appears that the `MongoDBLookupService` (or perhaps its `MongoDBControllerService`) stores Mongo document arrays as `ArrayList`s, which `DataTypeUtils#toArray` does not support. A simple fix appears to be to add a few lines to `DataTypeUtils#toArray` to support `ArrayList`s. I will provide a PR for this issue.

      This issue appears in 1.7.1, and is extant in 1.8.0-SNAPSHOT (2018-10-02 b4c8e0179). I am not aware of any workaround to enrich a Record with an array from Mongo.

      LookupRecord Configuration

      Property Value
      Result RecordPath /f
      Record Result Contents Insert Entire Record
      _id /_id

      MongoDBLookupService Configuration

      Property Value
      Lookup Value Field f

      Conversion error (with irrelevant Mongo document and schema properties elided):

      2018-10-04 15:00:31,348 ERROR [Timer-Driven Process Thread-1] o.a.n.processors.standard.LookupRecord LookupRecord[id=339bf72c-91c3-36f0-ac14-bc8b734e2582] Failed to write MapRecord[{... f=[Document{{fid=5762addaffa7cf0980267acb, l=1.0}}, Document{{fid=5762adf32fc74d0418104201, l=2.12}}, Document{{fid=5762addd18b8b607802fec6f, l=3.32}}, Document{{fid=587542dd638e870838a79d08, l=1.0}}, Document{{fid=5762addbffa7cf0980267acd, l=3.32}}], i=[Ljava.lang.Object;@35dee8fc, ...] with schema {
        "namespace": "nifi",
        "name": "MongoPosting",
        "type": "record",
        "fields": [
          ...
          { "name": "f", "type": [
              "null",
              { "type": "array", "items": {
                  "type": "record", "name": "PostingFeatureDetails", "fields": [
                      { "name": "fid", "type": "string" },
                      { "name": "l" , "type": ["null", "double"] }
                  ]}
              }
          ]},
          ...
        ]
      } as a JSON Object due to org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}}, Document{{fid=5762adf32fc74d0418104201, l=2.12}}, Document{{fid=5762addd18b8b607802fec6f, l=3.32}}, Document{{fid=587542dd638e870838a79d08, l=1.0}}, Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class java.util.ArrayList to Object Array for field f: org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}}, Document{{fid=5762adf32fc74d0418104201, l=2.12}}, Document{{fid=5762addd18b8b607802fec6f, l=3.32}}, Document{{fid=587542dd638e870838a79d08, l=1.0}}, Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class java.util.ArrayList to Object Array for field f
      org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}}, Document{{fid=5762adf32fc74d0418104201, l=2.12}}, Document{{fid=5762addd18b8b607802fec6f, l=3.32}}, Document{{fid=587542dd638e870838a79d08, l=1.0}}, Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class java.util.ArrayList to Object Array for field f
          at org.apache.nifi.serialization.record.util.DataTypeUtils.toArray(DataTypeUtils.java:347)
          at org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:153)
          at org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:115)
          at org.apache.nifi.json.WriteJsonResult.writeValue(WriteJsonResult.java:284)
          at org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:187)
          at org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:136)
          at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
          at org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:148)
          at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2218)
          at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2186)
          at org.apache.nifi.processors.standard.AbstractRouteRecord.onTrigger(AbstractRouteRecord.java:121)
          at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
          at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
          at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
          at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)
      2018-10-04 15:00:31,349 ERROR [Timer-Driven Process Thread-1] o.a.n.processors.standard.LookupRecord LookupRecord[id=339bf72c-91c3-36f0-ac14-bc8b734e2582] Failed to process StandardFlowFileRecord[uuid=29b498fe-787f-4c62-a237-f95a8ebcb0b2,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1538665215854-2, container=default, section=2], offset=100486, length=635840],offset=624238,name=1675395032391,size=11602]: org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}}, Document{{fid=5762adf32fc74d0418104201, l=2.12}}, Document{{fid=5762addd18b8b607802fec6f, l=3.32}}, Document{{fid=587542dd638e870838a79d08, l=1.0}}, Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class java.util.ArrayList to Object Array for field f
      org.apache.nifi.serialization.record.util.IllegalTypeConversionException: Cannot convert value [[Document{{fid=5762addaffa7cf0980267acb, l=1.0}}, Document{{fid=5762adf32fc74d0418104201, l=2.12}}, Document{{fid=5762addd18b8b607802fec6f, l=3.32}}, Document{{fid=587542dd638e870838a79d08, l=1.0}}, Document{{fid=5762addbffa7cf0980267acd, l=3.32}}]] of type class java.util.ArrayList to Object Array for field f
          at org.apache.nifi.serialization.record.util.DataTypeUtils.toArray(DataTypeUtils.java:347)
          at org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:153)
          at org.apache.nifi.serialization.record.util.DataTypeUtils.convertType(DataTypeUtils.java:115)
          at org.apache.nifi.json.WriteJsonResult.writeValue(WriteJsonResult.java:284)
          at org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:187)
          at org.apache.nifi.json.WriteJsonResult.writeRecord(WriteJsonResult.java:136)
          at org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
          at org.apache.nifi.processors.standard.AbstractRouteRecord$1.process(AbstractRouteRecord.java:148)
          at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2218)
          at org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2186)
          at org.apache.nifi.processors.standard.AbstractRouteRecord.onTrigger(AbstractRouteRecord.java:121)
          at org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
          at org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1165)
          at org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:203)
          at org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
          at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
          at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
          at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
          at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
          at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
          at java.lang.Thread.run(Thread.java:748)

      Attachments

        Issue Links

        Activity

          githubbot ASF GitHub Bot added a comment -

          GitHub user snagacarl opened a pull request:

          https://github.com/apache/nifi/pull/3049

          NIFI-5664 Support ArrayList in DataTypeUtils#toArray

          Thank you for submitting a contribution to Apache NiFi.

          In order to streamline the review of the contribution we ask you
          to ensure the following steps have been taken:

              1. For all changes:
          • [Y] Is there a JIRA ticket associated with this PR? Is it referenced
            in the commit message?
          • [Y] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
          • [Y] Has your PR been rebased against the latest commit within the target branch (typically master)?
          • [Y] Is your initial contribution a single, squashed commit?
              1. For code changes:
          • [Y] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
          • [Y] Have you written or updated unit tests to verify your changes?
          • [N/A] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
          • [N/A] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
          • [N/A] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
          • [N/A] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
              1. For documentation related changes:
          • [N/A] Have you ensured that format looks appropriate for the output in which it is rendered?
              1. Note:
                Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/Snagajob/nifi NIFI-5664

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/3049.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #3049


          commit 30d0410f8f35cff772cc74fc36e4368d33310aec
          Author: Carl Gieringer <carl.gieringer@...>
          Date: 2018-10-04T16:50:08Z

          NIFI-5664 Support ArrayList in DataTypeUtils#toArray


          githubbot ASF GitHub Bot added a comment - GitHub user snagacarl opened a pull request: https://github.com/apache/nifi/pull/3049 NIFI-5664 Support ArrayList in DataTypeUtils#toArray Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: For all changes: [Y] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [Y] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [Y] Has your PR been rebased against the latest commit within the target branch (typically master)? [Y] Is your initial contribution a single, squashed commit? For code changes: [Y] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [Y] Have you written or updated unit tests to verify your changes? [N/A] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0] ( http://www.apache.org/legal/resolved.html#category-a)? [N/A] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? [N/A] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? [N/A] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? For documentation related changes: [N/A] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/Snagajob/nifi NIFI-5664 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/3049.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #3049 commit 30d0410f8f35cff772cc74fc36e4368d33310aec Author: Carl Gieringer <carl.gieringer@...> Date: 2018-10-04T16:50:08Z NIFI-5664 Support ArrayList in DataTypeUtils#toArray
          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on the issue: https://github.com/apache/nifi/pull/3049 Apache issue: https://issues.apache.org/jira/browse/NIFI-5664
          githubbot ASF GitHub Bot added a comment -

          Github user ottobackwards commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223141579

          — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java —
          @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value)

          { return dest; }

          — End diff –

          Why wouldn't this be any List implementation?

          githubbot ASF GitHub Bot added a comment - Github user ottobackwards commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223141579 — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java — @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value) { return dest; } — End diff – Why wouldn't this be any List implementation?
          githubbot ASF GitHub Bot added a comment -

          Github user MikeThomsen commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223181320

          — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java —
          @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value)

          { return dest; }

          + if (value instanceof ArrayList) {
          — End diff –

          I agree with @ottobackwards. It really should not be a specific implementation.

          githubbot ASF GitHub Bot added a comment - Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223181320 — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java — @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value) { return dest; } + if (value instanceof ArrayList) { — End diff – I agree with @ottobackwards. It really should not be a specific implementation.
          githubbot ASF GitHub Bot added a comment -

          Github user MikeThomsen commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223181332

          — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java —
          @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() {

          }

          + @Test
          + public void testToArray() {
          + final ArrayList<String> arrayList = new ArrayList<>();
          — End diff –

          Should be `final List<String>....`

          githubbot ASF GitHub Bot added a comment - Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223181332 — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java — @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() { } + @Test + public void testToArray() { + final ArrayList<String> arrayList = new ArrayList<>(); — End diff – Should be `final List<String>....`
          githubbot ASF GitHub Bot added a comment -

          Github user MikeThomsen commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223181352

          — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java —
          @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() {

          }

          + @Test
          + public void testToArray() {
          + final ArrayList<String> arrayList = new ArrayList<>();
          + arrayList.add("Seven");
          — End diff –

          This could also be reduced to `final list<String> arrayList = Arrays.asList("Seven", "Eleven", "Thirteen");`

          githubbot ASF GitHub Bot added a comment - Github user MikeThomsen commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223181352 — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java — @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() { } + @Test + public void testToArray() { + final ArrayList<String> arrayList = new ArrayList<>(); + arrayList.add("Seven"); — End diff – This could also be reduced to `final list<String> arrayList = Arrays.asList("Seven", "Eleven", "Thirteen");`
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on the issue:

          https://github.com/apache/nifi/pull/3049

          Great suggestions. I can incorporate and re-test this week. Thanks for your feedback!

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on the issue: https://github.com/apache/nifi/pull/3049 Great suggestions. I can incorporate and re-test this week. Thanks for your feedback!
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223393416

          — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java —
          @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value)

          { return dest; }

          — End diff –

          I have updated the PR. Thanks for the good idea!

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223393416 — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java — @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value) { return dest; } — End diff – I have updated the PR. Thanks for the good idea!
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223396193

          — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java —
          @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value)

          { return dest; }

          + if (value instanceof ArrayList) {
          — End diff –

          Great idea. I have updated.

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223396193 — Diff: nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java — @@ -339,6 +339,11 @@ public static boolean isRecordTypeCompatible(final Object value) { return dest; } + if (value instanceof ArrayList) { — End diff – Great idea. I have updated.
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223396314

          — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java —
          @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() {

          }

          + @Test
          + public void testToArray() {
          + final ArrayList<String> arrayList = new ArrayList<>();
          — End diff –

          Good catch. Updated in the PR now.

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223396314 — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java — @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() { } + @Test + public void testToArray() { + final ArrayList<String> arrayList = new ArrayList<>(); — End diff – Good catch. Updated in the PR now.
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on a diff in the pull request:

          https://github.com/apache/nifi/pull/3049#discussion_r223396416

          — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java —
          @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() {

          }

          + @Test
          + public void testToArray() {
          + final ArrayList<String> arrayList = new ArrayList<>();
          + arrayList.add("Seven");
          — End diff –

          I have updated the PR. Thanks!

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on a diff in the pull request: https://github.com/apache/nifi/pull/3049#discussion_r223396416 — Diff: nifi-commons/nifi-record/src/test/java/org/apache/nifi/serialization/record/TestDataTypeUtils.java — @@ -168,6 +168,21 @@ public void testConvertRecordFieldToObject() { } + @Test + public void testToArray() { + final ArrayList<String> arrayList = new ArrayList<>(); + arrayList.add("Seven"); — End diff – I have updated the PR. Thanks!
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on the issue:

          https://github.com/apache/nifi/pull/3049

          I rebased onto master, FYI.

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on the issue: https://github.com/apache/nifi/pull/3049 I rebased onto master, FYI.
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on the issue:

          https://github.com/apache/nifi/pull/3049

          Now that the checks have passed, I believe this PR is G2G as far as comments so far are concerned.

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on the issue: https://github.com/apache/nifi/pull/3049 Now that the checks have passed, I believe this PR is G2G as far as comments so far are concerned.
          githubbot ASF GitHub Bot added a comment -

          Github user ottobackwards commented on the issue:

          https://github.com/apache/nifi/pull/3049

          This looks great to me, I thought there was a specific use case for this, if so, it might be good to have a unit or integration text that showed that case.

          githubbot ASF GitHub Bot added a comment - Github user ottobackwards commented on the issue: https://github.com/apache/nifi/pull/3049 This looks great to me, I thought there was a specific use case for this, if so, it might be good to have a unit or integration text that showed that case.
          githubbot ASF GitHub Bot added a comment -

          Github user MikeThomsen commented on the issue:

          https://github.com/apache/nifi/pull/3049

          I'll try to review tonight. Been pretty sick. Thanks for the updates @snagacarl

          githubbot ASF GitHub Bot added a comment - Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3049 I'll try to review tonight. Been pretty sick. Thanks for the updates @snagacarl
          githubbot ASF GitHub Bot added a comment -

          Github user asfgit closed the pull request at:

          https://github.com/apache/nifi/pull/3049

          githubbot ASF GitHub Bot added a comment - Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/3049
          githubbot ASF GitHub Bot added a comment -

          Github user MikeThomsen commented on the issue:

          https://github.com/apache/nifi/pull/3049

          @snagacarl since this was for the Mongo lookup service, wanted to make sure that you're aware that in 1.8 there will be a slight breaking change. The "configuration" controller service and the lookup service are now cleanly separated so that the lookup service can be cheerfully ignorant of connection details like the URL, SSL and authentication details. That way plugging in a new cluster or something will be faster and easier for Mongo users.

          githubbot ASF GitHub Bot added a comment - Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3049 @snagacarl since this was for the Mongo lookup service, wanted to make sure that you're aware that in 1.8 there will be a slight breaking change. The "configuration" controller service and the lookup service are now cleanly separated so that the lookup service can be cheerfully ignorant of connection details like the URL, SSL and authentication details. That way plugging in a new cluster or something will be faster and easier for Mongo users.
          githubbot ASF GitHub Bot added a comment -

          Github user snagacarl commented on the issue:

          https://github.com/apache/nifi/pull/3049

          Yes, I did test the changes in 1.8 and I noticed that change to how the MongoDBLookupService uses a MongoDBControllerService. Thanks for the heads up!

          githubbot ASF GitHub Bot added a comment - Github user snagacarl commented on the issue: https://github.com/apache/nifi/pull/3049 Yes, I did test the changes in 1.8 and I noticed that change to how the MongoDBLookupService uses a MongoDBControllerService. Thanks for the heads up!
          githubbot ASF GitHub Bot added a comment -

          Github user MikeThomsen commented on the issue:

          https://github.com/apache/nifi/pull/3049

          Cool. FYI, the long term goal is to make controller services the exclusive source of configurations for configurations. That's because a single `MongoClient` is really a connection pool and is-according to Mongo Inc.-thread-safe and intended to be used across an entire app. That should help Mongo users tighten up on how many connections they have open from NiFi, especially a cluster (since each node has N number of MongoClients where N = # of mongo processors).

          githubbot ASF GitHub Bot added a comment - Github user MikeThomsen commented on the issue: https://github.com/apache/nifi/pull/3049 Cool. FYI, the long term goal is to make controller services the exclusive source of configurations for configurations. That's because a single `MongoClient` is really a connection pool and is- according to Mongo Inc. -thread-safe and intended to be used across an entire app. That should help Mongo users tighten up on how many connections they have open from NiFi, especially a cluster (since each node has N number of MongoClients where N = # of mongo processors).

          People

            Unassigned Unassigned
            carlgieringer Carl Gieringer
            Votes:
            0 Vote for this issue
            Watchers:
            Stop watching this issue

            Dates

              Created:
              Updated:
              Resolved:

              Slack