We often hear from users that they want to know when all files in a directory (or a GCS Bucket or AWS bucket, etc.) have finished some sort of processing. This is difficult to accomplish with the existing processors.
NIFI-7476 makes it so that a Process Group is able to process a single FlowFile at a time and optionally wait until all FlowFiles have finished processing before continuing on.
However, this still doesn't completely resolve this concern because the List* Processors create a single FlowFile per object listed. We need the ability to list all entities as a single FlowFile. Using a Record Writer allows us to accomplish this.