[HOP-3855] investigate options to optimize text file output when write to filename from field

• Type: Improvement
• Status: Closed
• Priority: Low
• Resolution: Fixed
• Fix Version/s: Migrated to GHI
• Component/s: Pipelines, Transforms
• Assignee: Unassigned
• Reporter: Bart Maertens
• Created: 22/Mar/22 13:16
• Updated: 17/Mar/23 13:10
• Resolved: 17/Mar/23 13:10

Description

Text File Output performance drops significantly when writing to multiple files simultaneously when accepting the filename from a field. 

Part of this is probably caused by 1 vs x I/O processes, but there could be room for improvement in the performance drop for this transform. 

 

tfo.hpl