Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12351

combine should be parallelizable in many cases

    XMLWordPrintableJSON

    Details

      Description

      Relevant discussion: https://lists.apache.org/thread.html/r9e7d9527eb1d4c9c097c91c010a25dabf4a5f8053d50dc3b6d90d36a%40%3Cdev.beam.apache.org%3E

      Currently we require Singleton partitioning for combine() because func might operate on the full dataset, but in many cases func is actually an elementwise method. We should detect this when possible (e.g. when func is an np.ufunc), and/or provide a flag to let the user indicate the function is elementwise.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              bhulette Brian Hulette
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: