Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-12456 Querying table in parallel  in JdbcIO
  3. BEAM-13847

Support dynamic partitioning of JdbcIO ranges via SDF

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • P2
    • Resolution: Unresolved
    • None
    • None
    • io-java-jdbc

    Description

      basically we'd turn ReadFn into an SDF that can dynamically partition the range it's currently reading.

      To commit this change, we'd need to performance-test its behavior, to make sure that duplicatiopn of work on the Database-side is not a problem when we subpartition a range that we're already querying.

      Attachments

        Issue Links

          is related to

          Sub-task - The sub-task of the issue BEAM-13846 Add a proper string partitioner for JdbcIO.readWithPartitions

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Open

          Activity

            People

              Unassigned Unassigned
              pabloem Pablo Estrada
              Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

                Created:
                Updated: