[BEAM-13686] OOM while logging a large pipeline even when logging level is higher

• Type: Bug
• Status: Resolved
• Priority: P1
• Resolution: Fixed
• Affects Version/s: 2.35.0
• Fix Version/s: 2.37.0
• Component/s: runner-dataflow
• Assignee: Vitaly Ivanov
• Reporter: Vitaly Ivanov
• Created: 19/Jan/22 08:14
• Updated: 04/Feb/22 23:22
• Resolved: 04/Feb/22 23:22

Description

Logging message is calculated even when logging level is higher, so there is no WA to increase the logging level. 

Exception in thread "main" java.lang.OutOfMemoryError: Java heap space
    at java.base/java.util.Arrays.copyOf(Arrays.java:3745)
    at java.base/java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:172)
    at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:538)
    at java.base/java.lang.StringBuilder.append(StringBuilder.java:174)
    at java.base/java.lang.StringBuilder.append(StringBuilder.java:85)
    at java.base/java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:573)
    at java.base/java.lang.StringBuilder.append(StringBuilder.java:204)
    at java.base/java.lang.StringBuilder.append(StringBuilder.java:85)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$TextGenerator.print(TextFormat.java:860)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printFieldValue(TextFormat.java:574)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printSingleField(TextFormat.java:743)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printField(TextFormat.java:443)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printMessage(TextFormat.java:705)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.print(TextFormat.java:353)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printFieldValue(TextFormat.java:597)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printSingleField(TextFormat.java:743)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printField(TextFormat.java:443)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printMessage(TextFormat.java:705)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.print(TextFormat.java:353)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printFieldValue(TextFormat.java:597)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printSingleField(TextFormat.java:743)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printField(TextFormat.java:435)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printMessage(TextFormat.java:705)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.print(TextFormat.java:353)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printFieldValue(TextFormat.java:597)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printSingleField(TextFormat.java:743)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printField(TextFormat.java:443)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printMessage(TextFormat.java:705)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.print(TextFormat.java:353)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.print(TextFormat.java:339)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat$Printer.printToString(TextFormat.java:606)
    at org.apache.beam.vendor.grpc.v1p36p0.com.google.protobuf.TextFormat.printToString(TextFormat.java:149)

Issue Links

• links to: GitHub Pull Request #16559