Uploaded image for project: 'Beam'
  1. Beam
  2. BEAM-2523

GCP IO exposes protobuf on its API surface, causing user pain

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Open
    • Priority: P3
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Labels:
      None

      Description

      Putting the SDK, DataflowRunner, and GCP IO on the same classpath, results in (at least) three versions of protobuf getting pulled in. These should be made to converge. We should consider using maven enforcer, which I think can check this.

      [INFO] com.example:foo:jar:0.1
      [INFO] +- org.apache.beam:beam-sdks-java-core:jar:2.0.0:compile
      [INFO] +- org.apache.beam:beam-sdks-java-io-google-cloud-platform:jar:2.0.0:compile
      [INFO] |  +- org.apache.beam:beam-sdks-java-extensions-protobuf:jar:2.0.0:compile
      [INFO] |  |  \- (com.google.protobuf:protobuf-java:jar:3.2.0:compile - omitted for duplicate)
      [INFO] |  +- com.google.api.grpc:grpc-google-pubsub-v1:jar:0.1.0:compile
      [INFO] |  |  +- (com.google.protobuf:protobuf-java:jar:3.0.0:compile - omitted for conflict with 3.2.0)
      [INFO] |  |  \- com.google.api.grpc:grpc-google-iam-v1:jar:0.1.0:compile
      [INFO] |  |     \- (com.google.protobuf:protobuf-java:jar:3.0.0:compile - omitted for conflict with 3.2.0)
      [INFO] |  +- com.google.cloud.datastore:datastore-v1-proto-client:jar:1.4.0:compile
      [INFO] |  |  +- (com.google.cloud.datastore:datastore-v1-protos:jar:1.3.0:compile - omitted for duplicate)
      [INFO] |  |  +- (com.google.http-client:google-http-client:jar:1.20.0:compile - omitted for conflict with 1.22.0)
      [INFO] |  |  +- com.google.http-client:google-http-client-protobuf:jar:1.20.0:compile
      [INFO] |  |  |  +- (com.google.http-client:google-http-client:jar:1.20.0:compile - omitted for conflict with 1.22.0)
      [INFO] |  |  |  \- (com.google.protobuf:protobuf-java:jar:2.4.1:compile - omitted for conflict with 3.2.0)
      [INFO] |  +- com.google.cloud.datastore:datastore-v1-protos:jar:1.3.0:compile
      [INFO] |  |  +- (com.google.protobuf:protobuf-java:jar:3.0.0:compile - omitted for conflict with 3.2.0)
      [INFO] |  +- com.google.cloud.bigtable:bigtable-protos:jar:0.9.6.2:compile
      [INFO] |  |  +- (com.google.code.findbugs:jsr305:jar:3.0.1:compile - omitted for duplicate)
      [INFO] |  |  +- (com.google.protobuf:protobuf-java:jar:3.2.0:compile - omitted for duplicate)
      

      Incidentally, the dependency plugin stopped supporting the verbose tree, so we can't even visually inspect this except by downgrading.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                Unassigned
                Reporter:
                kenn Kenneth Knowles
              • Votes:
                2 Vote for this issue
                Watchers:
                6 Start watching this issue

                Dates

                • Created:
                  Updated: