[BEAM-6607] SchemaCoder cannot encode row with null value in array

• Type: Bug
• Status: Resolved
• Priority: P2
• Resolution: Fixed
• Fix Version/s: 2.11.0
• Component/s: sdk-java-core
• Labels: Done
• Assignee: Mike Pedersen
• Reporter: Mike Pedersen
• Created: 06/Feb/19 12:49
• Updated: 06/Oct/20 21:57
• Resolved: 08/Feb/19 00:00

Description

This example fails with a "cannot encode null Integer" exception:

import com.google.common.io.ByteStreams;
import org.apache.beam.sdk.schemas.Schema;
import org.apache.beam.sdk.schemas.SchemaCoder;
import org.apache.beam.sdk.values.Row;
import java.io.IOException;
import java.util.Collections;

public class Main {
    public static void main(String[] args) throws IOException {
        Schema schema = Schema.builder()
                .addField("a", Schema.FieldType.array(Schema.FieldType.INT32, true))
                .build();

        Row row = Row.withSchema(schema).addValue(Collections.singletonList(null)).build();

        SchemaCoder.of(schema).encode(row, ByteStreams.nullOutputStream());
    }
}

Note that null in the array should be OK, as the nullable parameter to Schema.FieldType.array is true.

An easy way of solving this could be to wrap inner coders with a NullableCoder, but a better way would probably to have something like a NullableIterableCoder that uses a bitset similarly to how the SchemaCoder encodes nullable fields.

I'll probably take a stab at fixing this and making a pull request.

Issue Links

• links to: GitHub Pull Request #7770