[GOBBLIN-1967] Add data node for generic external ingress egress

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: gobblin-service
• Assignee: Abhishek Tiwari
• Reporter: William Lo
• Created: 28/Nov/23 20:21
• Updated: 01/Dec/23 18:16

Description

GaaS has datanodes for HTTP, SFTP, and HDFS/Table varients. However, in some scenarios we want a generic identifier for an external ingress/egress flow running in GaaS. Reason being is that there are some usecases that can be very generic and we want to avoid adapting these flows to the opinionated setup of GaaS, while maintaining compatibility.