Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-9275

BIP-1: Beam Schema Options

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: In Progress
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: sdk-java-core
    • Labels:
      None

      Description

      Introduce the concept of Options in Beam Schema’s to add extra context to fields and schemas. In contrast to the current Beam metadata that is present in a FieldType, options would be added to fields, logical types and schemas. The schema convertors (ex. Avro, Proto, …) can add options/annotations/decorators that were in the original schema to the Beam schema with these options. These options, that add contextual metadata, can be used in the pipeline for specific transformations or augment the end schema in the target output.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                alexvanboxel Alex Van Boxel
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:

                  Time Tracking

                  Estimated:
                  Original Estimate - Not Specified
                  Not Specified
                  Remaining:
                  Remaining Estimate - 0h
                  0h
                  Logged:
                  Time Spent - 15h 50m
                  15h 50m