[BEAM-10673] DynamoDBIO.RetryConfiguration in AWS v2 is not correctly exposed

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Affects Version/s: 2.23.0
• Fix Version/s: 2.25.0
• Component/s: io-java-aws
• Assignee: Dennis Yung
• Reporter: Dennis Yung
• Created: 11/Aug/20 10:52
• Updated: 15/May/21 17:58
• Resolved: 15/May/21 17:58

Description

As described in the documentation, DynamoDBIO is initiated with a RetryConfiguration through withRetryConfiguration()

 PCollection<T> data = ...;
 data.apply(
           DynamoDBIO.<WriteRequest>write()
               .withWriteRequestMapperFn(
                   (SerializableFunction<T, KV<String, WriteRequest>>)
                       //Transforming your T data into KV<String, WriteRequest>
                       t -> KV.of(tableName, writeRequest))
               .withRetryConfiguration(
                    DynamoDBIO.RetryConfiguration.create(5, Duration.standardMinutes(1)))
               .withAwsClientsProvider(new BasicDynamoDbProvider(accessKey, secretKey, region));

It works on beam-sdks-java-io-amazon-web-services (aws sdk version 1), but not on beam-sdks-java-io-amazon-web-services2 (aws sdk version 2).

It is because io.aws2.dynamodb.DynamoDBIO.RetryConfiguration adopted a AutoValue builder design in place of the create method. However, the builder do not have public methods for setting values and building a RetryConfiguration. Moreover, validation was not implemented

  @AutoValue
  public abstract static class RetryConfiguration implements Serializable {
    
   /.../

    public static Builder builder() {
      return new AutoValue_DynamoDBIO_RetryConfiguration.Builder();
    }

    @AutoValue.Builder
    abstract static class Builder {
      abstract Builder setMaxAttempts(int maxAttempts);

      abstract Builder setMaxDuration(Duration maxDuration);

      abstract Builder setRetryPredicate(RetryPredicate retryPredicate);

      abstract RetryConfiguration build();
    }

Issue Links

• links to: GitHub Pull Request #12658