Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-10858

beam.io.jdbc not working on Google Cloud Dataflow

    XMLWordPrintableJSON

    Details

      Description

      Using the JDBC python wrapper to connect to an external server is failing on Cloud Dataflow.

      The error produced is;

      RuntimeError: /root/apache-beam-custom/packages/beam/sdks/java/extensions/schemaio-expansion-service/build/libs/beam-sdks-java-extensions-schemaio-expansion-service-2.25.0-SNAPSHOT.jar not found. Please build the server with cd /root/apache-beam-custom/packages/beam; ./gradlew :sdks:java:extensions:schemaio-expansion-service:shadowJar

      You can recreate the issue by running the below

      from apache_beam.io import jdbc
      jdbc.ReadFromJdbc(
       table_name='[tablename]',
       driver_class_name='com.mysql.cj.jdbc.Driver',
       jdbc_url='jdbc:mysql://[host]:3306/[databasename]',
       username='[username]',
       password='[password]')

       

        Attachments

          Activity

            People

            • Assignee:
              Unassigned
              Reporter:
              EhabElBadrawi Ehab El Badrawi
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated: