[BEAM-12085] PubsubLiteIO.read() fails in DataflowRunner v1 due to lack of BundleFinalizer support

• Type: New Feature
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.28.0
• Component/s: extensions-java-gcp
• Labels: PubsubLiteIO dataflow
• Assignee: Unassigned
• Reporter: Tianzi Cai
• Created: 01/Apr/21 23:02
• Updated: 04/Jun/22 20:06

Description

Reading from a Pub/Sub Lite subscription using PubsubLiteIO fails with DataflowRunner. It works in DirectRunner. 

import org.apache.beam.sdk.io.gcp.pubsublite.PubsubLiteIO;
//..
    pipeline
        .apply("Read From Lite", PubsubLiteIO.read(subscriberOpitons))
        .apply("Convert and print", MapElements.into(TypeDescriptors.strings()).via(
            (SequencedMessage sequencedMessage) -> {
              String data = sequencedMessage.getMessage().getData().toStringUtf8();
              LOG.info("Received: " + data);
              return data;
            }
        ));

java.lang.UnsupportedOperationException: BundleFinalizer unsupported by non-portable Dataflow.
	at org.apache.beam.runners.dataflow.worker.SplittableProcessFnFactory$SplittableDoFnRunnerFactory.lambda$createRunner$2(SplittableProcessFnFactory.java:170)
	at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.OutputAndTimeBoundedSplittableProcessElementInvoker$1.bundleFinalizer(OutputAndTimeBoundedSplittableProcessElementInvoker.java:195)
	at org.apache.beam.sdk.io.gcp.pubsublite.PerSubscriptionPartitionSdf$DoFnInvoker.invokeProcessElement(Unknown Source)
	at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.OutputAndTimeBoundedSplittableProcessElementInvoker.invokeProcessElement(OutputAndTimeBoundedSplittableProcessElementInvoker.java:123)
	at org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SplittableParDoViaKeyedWorkItems$ProcessFn.processElement(SplittableParDoViaKeyedWorkItems.java:523)