Uploaded image for project: 'Apache Hop'
  1. Apache Hop
  2. HOP-2783

Simple pipeline causes Metadata plugin registry error

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • High
    • Resolution: Not A Bug
    • 0.99
    • Not applicable
    • Beam, Hop Run
    • None

    Description

      I created a simple pipeline that reads a text file from a GCP bucket and outputs it into BQ table:

      File is located in a GCP bucket, accessible to the current credentials and I can preview the rows from it:

      File structure is very simple, one integer and two string columns:

      This is the root cause error that pops-up:

      Caused by: java.lang.RuntimeException: Error executing TransformFn org.apache.hop.beam.core.transform.TransformTransform$TransformFn.processElement(TransformTransform.java:521) Caused by: org.apache.hop.core.exception.HopException: Error reading metadata from JSON Error find metadata class for key pipeline-probe The metadata plugin for key pipeline-probe could not be found in the plugin registry org.apache.hop.core.metadata.SerializableMetadataProvider.<init>(SerializableMetadataProvider.java:137)

       

      ...
      java.util.concurrent.FutureTask.run(FutureTask.java:266) 
      java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149) 
      java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624) 
      java.lang.Thread.run(Thread.java:748) Caused by: java.lang.RuntimeException: Error executing TransformFn org.apache.hop.beam.core.transform.TransformTransform$TransformFn.processElement(TransformTransform.java:521) 
      Caused by: org.apache.hop.core.exception.HopException: Error reading metadata from JSON Error find metadata class for key pipeline-probe The metadata plugin for key pipeline-probe could not be found in the plugin registry 
      org.apache.hop.core.metadata.SerializableMetadataProvider.<init>(SerializableMetadataProvider.java:137) 
      org.apache.hop.beam.core.transform.TransformTransform$TransformFn.processElement(TransformTransform.java:289) 
      org.apache.hop.beam.core.transform.TransformTransform$TransformFn$DoFnInvoker.invokeProcessElement(Unknown Source) 
      org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:232) 
      org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) 
      org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:339) 
      org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) 
      org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49)
      

      Note: When I switch to  random generated rows (instead of text input), everything finishes ok.

       

      Attachments

        1. image-2021-04-12-23-22-27-272.png
          41 kB
          Goran Dumic
        2. image-2021-04-12-23-23-37-136.png
          27 kB
          Goran Dumic
        3. image-2021-04-12-23-24-50-904.png
          33 kB
          Goran Dumic
        4. image-2021-04-12-23-28-42-006.png
          9 kB
          Goran Dumic
        5. image-2021-04-12-23-28-49-829.png
          8 kB
          Goran Dumic
        6. image-2021-04-12-23-29-09-054.png
          19 kB
          Goran Dumic
        7. image-2021-04-12-23-30-41-555.png
          306 kB
          Goran Dumic

        Activity

          People

            Unassigned Unassigned
            gdumic Goran Dumic
            Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: