Uploaded image for project: 'Apache Nemo'
  1. Apache Nemo
  2. NEMO-298

Consider Processing DoFn PushBacks on Watermark

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:

      Description

      Optimize the case where we get many empty side inputs and want to avoid main input elements from waiting too long. 

      Processing DoFn PushBacks after updating the Watermark allows the waiting elements to acknowledge the empty views, at a cost of for-looping all of the pushed back elements.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              johnyangk John Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated: