[HOP-2783] Simple pipeline causes Metadata plugin registry error

• Type: Bug
• Status: Closed
• Priority: High
• Resolution: Not A Bug
• Affects Version/s: 0.99
• Fix Version/s: Not applicable
• Component/s: Beam, Hop Run
• Assignee: Unassigned
• Reporter: Goran Dumic
• Created: 12/Apr/21 21:33
• Updated: 01/May/21 17:42
• Resolved: 01/May/21 17:42

Description

I created a simple pipeline that reads a text file from a GCP bucket and outputs it into BQ table:

File is located in a GCP bucket, accessible to the current credentials and I can preview the rows from it:

File structure is very simple, one integer and two string columns:

This is the root cause error that pops-up:

Caused by: java.lang.RuntimeException: Error executing TransformFn org.apache.hop.beam.core.transform.TransformTransform$TransformFn.processElement(TransformTransform.java:521) Caused by: org.apache.hop.core.exception.HopException: Error reading metadata from JSON Error find metadata class for key pipeline-probe The metadata plugin for key pipeline-probe could not be found in the plugin registry org.apache.hop.core.metadata.SerializableMetadataProvider.<init>(SerializableMetadataProvider.java:137)

 

...
java.util.concurrent.FutureTask.run(FutureTask.java:266) 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error executing TransformFn org.apache.hop.beam.core.transform.TransformTransform$TransformFn.processElement(TransformTransform.java:521) 
Caused by: org.apache.hop.core.exception.HopException: Error reading metadata from JSON Error find metadata class for key pipeline-probe The metadata plugin for key pipeline-probe could not be found in the plugin registry 
org.apache.hop.core.metadata.SerializableMetadataProvider.<init>(SerializableMetadataProvider.java:137) 
org.apache.hop.beam.core.transform.TransformTransform$TransformFn.processElement(TransformTransform.java:289) 
org.apache.hop.beam.core.transform.TransformTransform$TransformFn$DoFnInvoker.invokeProcessElement(Unknown Source) 
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232) 
org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) 
org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339) 
org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) 
org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)

Note: When I switch to  random generated rows (instead of text input), everything finishes ok.