The Spark runner has more dependencies than it needs:
- The jackson_module_scala dependency is marked as compile but it is only needed at runtime so it ends up leaking this module and breaking compatibility for users providing Spark on scala 2.12.
- The dropwizard, scala-library and commons-compress dependencies are declared but unused.
- The Kafka dependency version is not aligned with the rest of Beam.
- The Kafka and zookeeper dependencies are used only for the tests but currently are provided