Uploaded image for project: 'Apache Hop (Retired)'
  1. Apache Hop (Retired)
  2. HOP-4194

GroupBy serialization error when running a workflow in a remote Hop server

Details

    • Bug
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 2.0.0
    • 2.1.0
    • Transforms
    • None

    Description

      A pipeline with a GroupBy transform fails with the following error when called as part of a workflow running in a remote hop server:

      Group by.0 - ERROR: Unexpected error
      Group by.0 - ERROR: org.apache.hop.core.exception.HopException:
      Group by.0 - Please specify an aggregation type for field 'field_name_agg'
      Group by.0 -
      Group by.0 -     at org.apache.hop.pipeline.transforms.groupby.GroupBy.newAggregate(GroupBy.java:622)
      Group by.0 -     at org.apache.hop.pipeline.transforms.groupby.GroupBy.processRow(GroupBy.java:168)
      Group by.0 -     at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:51)
      Group by.0 -     at java.base/java.lang.Thread.run(Thread.java:829)
      Group by.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)

      My guess it's a serialization error because the aggregation type attribute of the GroupBy action is set to '-' in the .zip that is sent to the remote hop server.

      Find attached a reproducer. The group-by-agg-fail.tar.gz contains a workflow W1, a pipeline P1 and an input file bod.csv.

      The script reproducer.sh list the necessary steps. It depends on environment variable HOP_CLIENT_INSTALL_DIR pointing to a Hop installation folder.

      As I commented in the users@ ML, the error is triggered by running the workflow remotely. In any other configuration (workflow locally, pipeline locally, pipeline remote) everything runs smoothly.

      Attachments

        1. groupby-agg-fail.tar.gz
          3 kB
          Maykel Moya
        2. reproducer.sh
          0.8 kB
          Maykel Moya

        Issue Links

          Activity

            People

              mcasters Matt Casters
              mmoyar Maykel Moya
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved:

                Slack

                  Issue deployment