Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4735

Make HBaseIO.read() based on SDF by default

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Open
    • P3
    • Resolution: Unresolved
    • None
    • None
    • io-java-hbase
    • None

    Description

      BEAM-4020 introduces HBaseIO reads based on SDF. So far the read() method still uses the Source based API because SDF does not support Dynamic Work Rebalancing (DWR) but the Source API of HBase does, so changing it means losing some functionality.

      Since DWR is only supported by Dataflow once Dataflow supports SDF  + DWR we can move the main read() function to use the SDF API and remove the Source based implementation. The rest of the runners already support Bounded based Reads (like HBase based on SDF does) via a default translation without DWR.

      Attachments

        Issue Links

          depends upon

          Sub-task - The sub-task of the issue BEAM-4737 SplittableDoFn dynamic rebalancing in Dataflow

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved
          is blocked by

          Sub-task - The sub-task of the issue BEAM-4287 SplittableDoFn: splitAtFraction() API for Java

          • P2 - Default priority. Will be triaged and planned according to community practices.
          • Resolved

          Activity

            People

              Unassigned Unassigned
              iemejia Ismaël Mejía
              Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

                Created:
                Updated: