[FLINK-36609] Protobuf Format not support proto3

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.20.0
• Fix Version/s: 1.20.2
• Component/s: Formats (JSON, Avro, Parquet, ORC, SequenceFile)
• Labels: pull-request-available
• Assignee: Hongshun Wang
• Reporter: Hongshun Wang
• Created: 28/Oct/24 06:17
• Updated: 17/Dec/24 19:11

Description

Currently, though ProtoToRowConverter try to handle PROTO3  as follows:

public ProtoToRowConverter(RowType rowType, PbFormatConfig formatConfig)
        throws PbCodegenException {
    try {
        Descriptors.Descriptor descriptor =
                PbFormatUtils.getDescriptor(formatConfig.getMessageClassName());
        Class<?> messageClass =
                Class.forName(
                        formatConfig.getMessageClassName(),
                        true,
                        Thread.currentThread().getContextClassLoader());
        String fullMessageClassName = PbFormatUtils.getFullJavaName(descriptor);
        boolean readDefaultValuesForPrimitiveTypes = formatConfig.isReadDefaultValues();
        if (descriptor.getFile().getSyntax() == Syntax.PROTO3) {
            // pb3 always read default values for primitive types
            readDefaultValuesForPrimitiveTypes = true;
        }
//...
} 

However, protoc will compile message of PROTO3 as 
FileDescriptor rather than Descriptor, thus the following code will throw exception:

public static Descriptors.Descriptor getDescriptor(String className) {
    try {
        Class<?> pbClass =
                Class.forName(className, true, Thread.currentThread().getContextClassLoader());
        return (Descriptors.Descriptor)
                pbClass.getMethod(PbConstant.PB_METHOD_GET_DESCRIPTOR).invoke(null);
    } catch (Exception e) {
        throw new IllegalArgumentException(
                String.format("get %s descriptors error!", className), e);
    }
} 

Issue Links

• links to: GitHub Pull Request #3726