Uploaded image for project: 'Flink'
  1. Flink
  2. FLINK-11275 Unified Catalog APIs
  3. FLINK-13313

create CatalogTableBuilder to support building CatalogTable from descriptors

    XMLWordPrintableJSON

    Details

      Description

      Found a usability issue.

      Previously, users can create an ExternalCatalogTable (deprecated) from descriptors via ExternalCatalogTableBuilder, and this helps smooth user experience of Flink Table API. E.g.

      ExternalCatalogTable table = ExternalCatalogTableBuilder(
      	new ExternalSystemXYZ()
      		.version("0.11"))
      		.withFormat(
      			new Json()
      				.jsonSchema("{...}")
      				.failOnMissingField(false))
      		。withSchema(
      			new Schema()
      				.field("user-name", "VARCHAR").from("u_name")
      				.field("count", "DECIMAL")
      		.supportsStreaming()
      		.asTableSource()
      
      oldCatalog.createTable("tble_name", table, false)
      

      If we don't have a builder to connect new CatalogTable and descriptor, how a user creates CatalogTable would be like the following example, which is quite inconvenient given users have to know all the key names.

      TableSchema schema = TableSchema.builder()
      	.field("name", DataTypes.STRING())
      	.field("age", DataTypes.INT())
      	.build();
      
      Map<String, String> properties = new HashMap<>();
      properties.put(CatalogConfig.IS_GENERIC, String.valueOf(true));
      properties.put("connector.type", "filesystem");
      properties.put("connector.path", "/tmp");
      properties.put("connector.property-version", "1");
      properties.put("update-mode", "append");
      
      properties.put("format.type", "csv");
      properties.put("format.property-version", "1");
      properties.put("format.fields.0.name", "name");
      properties.put("format.fields.0.type", "STRING");
      properties.put("format.fields.1.name", "age");
      properties.put("format.fields.1.type", "INT");
      
      ObjectPath path = new ObjectPath("mydb", "mytable");
      CatalogTable table = new CatalogTableImpl(schema, properties, "csv table");
      

      We need a similar new class CatalogTableBuilder for new Catalog APIs

      cc Tzu-Li (Gordon) Tai Kurt Young Xuefu Zhang

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                phoenixjiangnan Bowen Li
                Reporter:
                phoenixjiangnan Bowen Li
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 20m
                  20m