Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-4608

Improve / Fix documentation examples for Beam 2.4.0 for TextIO and BigQueryIO

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: 2.4.0
    • Fix Version/s: None
    • Component/s: io-java-gcp, io-java-text
    • Labels:
      None

      Description

      The documentation that is part of the code of TextIO has some usage examples.

      The description of how to achieve custom naming with the new FileIO in combination with the TextIO could be improved as the code in the "Advanced Features" is not really fully fleshed out and probably not in sync with how it should be done in Beam 2.4.0.

      https://beam.apache.org/documentation/sdks/javadoc/2.4.0/index.html?org/apache/beam/sdk/io/TextIO.html

      See SO Question: 

      https://stackoverflow.com/questions/50930409/how-to-replace-withfilenamepolicy-in-apache-beam-2-4

      The second part that could be improved is in the gcp.BigQueryIO class:

      https://github.com/apache/beam/blob/release-2.4.0/sdks/java/io/google-cloud-platform/src/main/java/org/apache/beam/sdk/io/gcp/bigquery/BigQueryIO.java#L217

      The example is also not fully fleshed out, but most importantly the
      BigQueryIO.write() should be replaced with BigQueryIO.<Quote>write() as otherwise it won't compile.

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              james-woods Tobias Kaymak
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated: