[HOP-4194] GroupBy serialization error when running a workflow in a remote Hop server

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 2.0.0
• Fix Version/s: 2.1.0
• Component/s: Transforms
• Assignee: Matt Casters
• Reporter: Maykel Moya
• Created: 05/Sep/22 13:41
• Updated: 03/Oct/22 14:40
• Resolved: 03/Oct/22 14:40

Description

A pipeline with a GroupBy transform fails with the following error when called as part of a workflow running in a remote hop server:

Group by.0 - ERROR: Unexpected error
Group by.0 - ERROR: org.apache.hop.core.exception.HopException:
Group by.0 - Please specify an aggregation type for field 'field_name_agg'
Group by.0 -
Group by.0 -     at org.apache.hop.pipeline.transforms.groupby.GroupBy.newAggregate(GroupBy.java:622)
Group by.0 -     at org.apache.hop.pipeline.transforms.groupby.GroupBy.processRow(GroupBy.java:168)
Group by.0 -     at org.apache.hop.pipeline.transform.RunThread.run(RunThread.java:51)
Group by.0 -     at java.base/java.lang.Thread.run(Thread.java:829)
Group by.0 - Finished processing (I=0, O=0, R=1, W=0, U=0, E=1)

My guess it's a serialization error because the aggregation type attribute of the GroupBy action is set to '-' in the .zip that is sent to the remote hop server.

Find attached a reproducer. The group-by-agg-fail.tar.gz contains a workflow W1, a pipeline P1 and an input file bod.csv.

The script reproducer.sh list the necessary steps. It depends on environment variable HOP_CLIENT_INSTALL_DIR pointing to a Hop installation folder.

As I commented in the users@ ML, the error is triggered by running the workflow remotely. In any other configuration (workflow locally, pipeline locally, pipeline remote) everything runs smoothly.

Issue Links

• links to: GitHub Pull Request #1711