[NIFI-5393] ReportLineageToAtlas fails for

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.5.0
• Fix Version/s: 1.8.0
• Labels: atlas lineage site-to-site
• Assignee: Koji Kawamura
• Reporter: Daniel Skates
• Created: 09/Jul/18 03:24
• Updated: 11/Jul/18 01:47
• Resolved: 11/Jul/18 01:47

Description

Description:

The ReportLineageToAtlas reporting task analyses provenance events and reports to Atlas.  Currently, the RAW protocol for Remote Processing Groups is not supported, only HTTP S2S protocol can be reported.

 

Relevant code:

 

Method: URL parseUrl(String url)

Class: org.apache.nifi.atlas.provenance.AbstractNiFiProvenanceEventAnalyzer

Bundle: nifi-atlas-bundle

https://github.com/apache/nifi/blob/79a7014a95dc3087f88248c732fb1e4ad8e6e128/nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/main/java/org/apache/nifi/atlas/provenance/AbstractNiFiProvenanceEventAnalyzer.java#L49

 

Relevant exception:

 

2018-06-11 05:00:11,407 ERROR [Timer-Driven Process Thread-18] o.a.n.a.reporting.ReportLineageToAtlas ReportLineageToAtlas[id=da4632be-779e-1cc3-b480-242c8c36e612] Skipping failed analyzing event ProvenanceEventRecord [eventId=7426097546, eventType=RECEIVE, eventTime=Mon Jun 11 04:47:36 AWST 2018, uuid=2ff2cb97-5011-47c9-b881-577989f21980, fileSize=10, componentId=de0df916-0163-1000-8497-cc7e7cd614d1, transitUri=nifi://nifi.example.com:8000/d57e73e1-fdd6-492b-9945-b6059df02976, sourceSystemFlowFileIdentifier=urn:nifi:d57e73e1-fdd6-492b-9945-b6059df02976, parentUuids=[], alternateIdentifierUri=null] due to java.lang.IllegalArgumentException: Failed to parse url nifi://nifi.example.com:8000/d57e73e1-fdd6-492b-9945-b6059df02976 due to java.net.MalformedURLException: unknown protocol: nifi.: java.lang.IllegalArgumentException: Failed to parse url nifi://nifi.example.com:8000/d57e73e1-fdd6-492b-9945-b6059df02976 due to java.net.MalformedURLException: unknown protocol: nifi
java.lang.IllegalArgumentException: Failed to parse url nifi://nifi.example.com:8000/d57e73e1-fdd6-492b-9945-b6059df02976 due to java.net.MalformedURLException: unknown protocol: nifi
 at org.apache.nifi.atlas.provenance.AbstractNiFiProvenanceEventAnalyzer.parseUrl(AbstractNiFiProvenanceEventAnalyzer.java:54)
 at org.apache.nifi.atlas.provenance.analyzer.NiFiS2S.parseTransitURL(NiFiS2S.java:36)
 at org.apache.nifi.atlas.provenance.analyzer.NiFiRootGroupPort.analyze(NiFiRootGroupPort.java:57)
 at org.apache.nifi.atlas.provenance.lineage.AbstractLineageStrategy.executeAnalyzer(AbstractLineageStrategy.java:70)
 at org.apache.nifi.atlas.provenance.lineage.SimpleFlowPathLineage.processEvent(SimpleFlowPathLineage.java:41)
 at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.lambda$consumeNiFiProvenanceEvents$8(ReportLineageToAtlas.java:666)
 at org.apache.nifi.reporting.util.provenance.ProvenanceEventConsumer.consumeEvents(ProvenanceEventConsumer.java:204)
 at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.consumeNiFiProvenanceEvents(ReportLineageToAtlas.java:663)
 at org.apache.nifi.atlas.reporting.ReportLineageToAtlas.onTrigger(ReportLineageToAtlas.java:615)
 at org.apache.nifi.controller.tasks.ReportingTaskWrapper.run(ReportingTaskWrapper.java:41)
 at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
 at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
 at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
 at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
 at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
 at java.lang.Thread.run(Thread.java:748)
Caused by: java.net.MalformedURLException: unknown protocol: nifi
 at java.net.URL.<init>(URL.java:600)
 at java.net.URL.<init>(URL.java:490)
 at java.net.URL.<init>(URL.java:439)
 at org.apache.nifi.atlas.provenance.AbstractNiFiProvenanceEventAnalyzer.parseUrl(AbstractNiFiProvenanceEventAnalyzer.java:51)
 ... 16 common frames omitted

 

Issue Links

• Is contained by: NIFI-4654 RAW S2S transit URI should contain Port ID instead of FlowFile ID