[BEAM-9424] Grouping By LogicalTypes is not supported

• Type: Bug
• Status: Open
• Priority: P3
• Resolution: Unresolved
• Affects Version/s: 2.19.0
• Component/s: sdk-java-core
• Labels: Clarified sql
• Assignee: Unassigned
• Reporter: fdiazgon
• Created: 02/Mar/20 15:53
• Updated: 04/Jun/22 15:26

Description

Creating a schema from a BQ schema that has either TIME, DATE or DATETIME columns, and grouping by one of these fields throws NullPointerException.

Pipeline pipeline = Pipeline.create();

Schema beamSchemaWithLogicalTypes =
    BigQueryUtils.fromTableSchema(
        new TableSchema()
            .setFields(
                Arrays.asList(
                    new TableFieldSchema().setName("fTime").setType("TIME"),
                    new TableFieldSchema().setName("fDate").setType("DATE"),
                    new TableFieldSchema().setName("fDatetime").setType("DATETIME"))));

Row row =
    Row.withSchema(beamSchemaWithLogicalTypes)
        .addValues(
            DateTime.parse("2020-02-02"),
            DateTime.parse("2020-02-02"),
            DateTime.parse("2020-02-02T00:00:00"))
        .build();

PCollection<Row> outputRow =
    pipeline
        .apply(Create.of(row))
        .setRowSchema(beamSchemaWithLogicalTypes)
        .apply(
            SqlTransform.query(
                "SELECT fTime, fDate, fDatetime FROM PCOLLECTION GROUP BY fTime, fDate, fDatetime"));

pipeline.run();

Issue Links

• links to: GitHub Pull Request #11015
• links to: GitHub Pull Request #11034