Details

    • Type: Improvement
    • Status: Patch Available
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: 1.3.0
    • Fix Version/s: 1.4.0
    • Component/s: Core Framework
    • Labels:
      None

      Description

      We should improve the template loading code (uses JAXBContext).

        Issue Links

          Activity

          Hide
          githubbot ASF GitHub Bot added a comment -

          GitHub user alopresto opened a pull request:

          https://github.com/apache/nifi/pull/2134

          NIFI-4357 Global improvement of XML unmarshalling

          Thank you for submitting a contribution to Apache NiFi.

          In order to streamline the review of the contribution we ask you
          to ensure the following steps have been taken:

              1. For all changes:
          • [x] Is there a JIRA ticket associated with this PR? Is it referenced
            in the commit message?
          • [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character.
          • [x] Has your PR been rebased against the latest commit within the target branch (typically master)?
          • [ ] Is your initial contribution a single, squashed commit?
              1. For code changes:
          • [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder?
          • [x] Have you written or updated unit tests to verify your changes?
          • [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)?
          • [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly?
          • [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly?
          • [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties?
              1. For documentation related changes:
          • [ ] Have you ensured that format looks appropriate for the output in which it is rendered?
              1. Note:
                Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible.

          You can merge this pull request into a Git repository by running:

          $ git pull https://github.com/alopresto/nifi NIFI-4357

          Alternatively you can review and apply these changes as the patch at:

          https://github.com/apache/nifi/pull/2134.patch

          To close this pull request, make a commit to your master/trunk branch
          with (at least) the following in the commit message:

          This closes #2134


          commit 0fa6ca17629094845bc9e374f442e3617205f1fd
          Author: Andy LoPresto <alopresto@apache.org>
          Date: 2017-09-06T23:45:13Z

          NIFI-4353 Added XmlUtils class.
          Added unit test.
          Added XXE test resource.

          commit 91ff58d038d3afe6a6c1aa13226a2c3050612938
          Author: Andy LoPresto <alopresto@apache.org>
          Date: 2017-09-07T18:57:33Z

          NIFI-4357 Refactored JAXB unmarshalling globally to prevent XXE attacks.
          Refactored duplicated/legacy code.

          commit f2b396eb629f3adadd56eaff86666ce9ee245426
          Author: Andy LoPresto <alopresto@apache.org>
          Date: 2017-09-07T19:03:35Z

          NIFI-4357 Cleaned up commented code.
          Switched from FileInputStream back to StreamSource in AuthorizerFactoryBean.


          Show
          githubbot ASF GitHub Bot added a comment - GitHub user alopresto opened a pull request: https://github.com/apache/nifi/pull/2134 NIFI-4357 Global improvement of XML unmarshalling Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: For all changes: [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? [x] Does your PR title start with NIFI-XXXX where XXXX is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. [x] Has your PR been rebased against the latest commit within the target branch (typically master)? [ ] Is your initial contribution a single, squashed commit? For code changes: [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? [x] Have you written or updated unit tests to verify your changes? [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0] ( http://www.apache.org/legal/resolved.html#category-a)? [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? For documentation related changes: [ ] Have you ensured that format looks appropriate for the output in which it is rendered? Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/alopresto/nifi NIFI-4357 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/2134.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #2134 commit 0fa6ca17629094845bc9e374f442e3617205f1fd Author: Andy LoPresto <alopresto@apache.org> Date: 2017-09-06T23:45:13Z NIFI-4353 Added XmlUtils class. Added unit test. Added XXE test resource. commit 91ff58d038d3afe6a6c1aa13226a2c3050612938 Author: Andy LoPresto <alopresto@apache.org> Date: 2017-09-07T18:57:33Z NIFI-4357 Refactored JAXB unmarshalling globally to prevent XXE attacks. Refactored duplicated/legacy code. commit f2b396eb629f3adadd56eaff86666ce9ee245426 Author: Andy LoPresto <alopresto@apache.org> Date: 2017-09-07T19:03:35Z NIFI-4357 Cleaned up commented code. Switched from FileInputStream back to StreamSource in AuthorizerFactoryBean.
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mcgilman commented on the issue:

          https://github.com/apache/nifi/pull/2134

          Will review...

          Show
          githubbot ASF GitHub Bot added a comment - Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2134 Will review...
          Hide
          githubbot ASF GitHub Bot added a comment -

          Github user mcgilman commented on the issue:

          https://github.com/apache/nifi/pull/2134

          Thanks for the PR @alopresto! I've run into a small issue with one of my tests. When I upload a template into my NiFi instance subsequent restarts fail with the following exception:

          ```
          org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.RuntimeException: Could not parse XML as a valid template
          at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:436)
          at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1591)
          at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84)
          at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:723)
          at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:534)
          at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72)
          at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876)
          at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532)
          at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839)
          at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344)
          at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480)
          at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442)
          at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799)
          at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261)
          at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540)
          at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
          at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
          at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113)
          at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
          at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
          at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
          at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
          at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
          at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290)
          at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
          at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131)
          at org.eclipse.jetty.server.Server.start(Server.java:452)
          at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105)
          at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113)
          at org.eclipse.jetty.server.Server.doStart(Server.java:419)
          at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
          at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:707)
          at org.apache.nifi.NiFi.<init>(NiFi.java:160)
          at org.apache.nifi.NiFi.main(NiFi.java:268)
          Caused by: java.lang.RuntimeException: Could not parse XML as a valid template
          at org.apache.nifi.controller.TemplateUtils.parseDto(TemplateUtils.java:51)
          at org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1384)
          at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:322)
          ... 33 common frames omitted
          Caused by: java.lang.UnsupportedOperationException: Cannot create XMLStreamReader or XMLEventReader from a javax.xml.transform.dom.DOMSource
          at com.sun.xml.internal.stream.XMLInputFactoryImpl.jaxpSourcetoXMLInputSource(XMLInputFactoryImpl.java:302)
          at com.sun.xml.internal.stream.XMLInputFactoryImpl.createXMLStreamReader(XMLInputFactoryImpl.java:143)
          at org.apache.nifi.security.xml.XmlUtils.createSafeReader(XmlUtils.java:43)
          at org.apache.nifi.persistence.TemplateDeserializer.deserialize(TemplateDeserializer.java:41)
          at org.apache.nifi.controller.TemplateUtils.parseDto(TemplateUtils.java:49)
          ... 35 common frames omitted
          ```

          Show
          githubbot ASF GitHub Bot added a comment - Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/2134 Thanks for the PR @alopresto! I've run into a small issue with one of my tests. When I upload a template into my NiFi instance subsequent restarts fail with the following exception: ``` org.apache.nifi.controller.serialization.FlowSynchronizationException: java.lang.RuntimeException: Could not parse XML as a valid template at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:436) at org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1591) at org.apache.nifi.persistence.StandardXMLFlowConfigurationDAO.load(StandardXMLFlowConfigurationDAO.java:84) at org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:723) at org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:534) at org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:72) at org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:876) at org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:532) at org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:839) at org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:344) at org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1480) at org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1442) at org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:799) at org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:261) at org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:540) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:113) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.server.handler.gzip.GzipHandler.doStart(GzipHandler.java:290) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:131) at org.eclipse.jetty.server.Server.start(Server.java:452) at org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:105) at org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:113) at org.eclipse.jetty.server.Server.doStart(Server.java:419) at org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68) at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:707) at org.apache.nifi.NiFi.<init>(NiFi.java:160) at org.apache.nifi.NiFi.main(NiFi.java:268) Caused by: java.lang.RuntimeException: Could not parse XML as a valid template at org.apache.nifi.controller.TemplateUtils.parseDto(TemplateUtils.java:51) at org.apache.nifi.controller.StandardFlowSynchronizer.addProcessGroup(StandardFlowSynchronizer.java:1384) at org.apache.nifi.controller.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:322) ... 33 common frames omitted Caused by: java.lang.UnsupportedOperationException: Cannot create XMLStreamReader or XMLEventReader from a javax.xml.transform.dom.DOMSource at com.sun.xml.internal.stream.XMLInputFactoryImpl.jaxpSourcetoXMLInputSource(XMLInputFactoryImpl.java:302) at com.sun.xml.internal.stream.XMLInputFactoryImpl.createXMLStreamReader(XMLInputFactoryImpl.java:143) at org.apache.nifi.security.xml.XmlUtils.createSafeReader(XmlUtils.java:43) at org.apache.nifi.persistence.TemplateDeserializer.deserialize(TemplateDeserializer.java:41) at org.apache.nifi.controller.TemplateUtils.parseDto(TemplateUtils.java:49) ... 35 common frames omitted ```

            People

            • Assignee:
              alopresto Andy LoPresto
              Reporter:
              alopresto Andy LoPresto
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:

                Development