[HUDI-4568] Cloudwatch reporter not being created with hudi-spark-bundle and hudi-aws-bundle in classpath

• Type: Bug
• Status: Closed
• Priority: Blocker
• Resolution: Fixed
• Fix Version/s: 0.12.0
• Labels: pull-request-available
• Story Points: 2
• Assignee: Sagar Sumit
• Reporter: Sagar Sumit
• Created: 08/Aug/22 07:15
• Updated: 23/Aug/22 15:20
• Resolved: 09/Aug/22 16:14

Description

spark-shell --jars hudi-spark3-bundle_2.12-0.13.0-SNAPSHOT.jar,hudi-aws-bundle-0.13.0-SNAPSHOT.jar --conf "spark.serializer=org.apache.spark.serializer.KryoSerializer" --conf "spark.sql.catalog.spark_catalog=org.apache.spark.sql.hudi.catalog.HoodieCatalog" --conf "spark.sql.extensions=org.apache.spark.sql.hudi.HoodieSparkSessionExtension"

/**

• DO df.write.format("hudi").options(**common_opts).option("hoodie.metrics.on", "true").option("hoodie.metrics.reporter.type", "CLOUDWATCH").save()
  **/

java.lang.NoSuchMethodError: org.apache.hudi.aws.cloudwatch.CloudWatchReporter.forRegistry(Lorg/apache/hudi/com/codahale/metrics/MetricRegistry;)Lorg/apache/hudi/aws/cloudwatch/CloudWatchReporter$Builder;
at org.apache.hudi.metrics.cloudwatch.CloudWatchMetricsReporter.createCloudWatchReporter(CloudWatchMetricsReporter.java:57)
at org.apache.hudi.metrics.cloudwatch.CloudWatchMetricsReporter.<init>(CloudWatchMetricsReporter.java:47)
at org.apache.hudi.metrics.MetricsReporterFactory.createReporter(MetricsReporterFactory.java:82)
at org.apache.hudi.metrics.Metrics.<init>(Metrics.java:50)
at org.apache.hudi.metrics.Metrics.init(Metrics.java:96)
at org.apache.hudi.metrics.HoodieMetrics.<init>(HoodieMetrics.java:61)
at org.apache.hudi.client.BaseHoodieWriteClient.<init>(BaseHoodieWriteClient.java:178)
at org.apache.hudi.client.SparkRDDWriteClient.<init>(SparkRDDWriteClient.java:95)
at org.apache.hudi.client.SparkRDDWriteClient.<init>(SparkRDDWriteClient.java:79)
at org.apache.hudi.DataSourceUtils.createHoodieClient(DataSourceUtils.java:193)
at org.apache.hudi.HoodieSparkSqlWriter$.$anonfun$write$13(HoodieSparkSqlWriter.scala:311)
at scala.Option.getOrElse(Option.scala:189)
at org.apache.hudi.HoodieSparkSqlWriter$.write(HoodieSparkSqlWriter.scala:310)
at org.apache.hudi.DefaultSource.createRelation(DefaultSource.scala:183)

Issue Links

• links to: GitHub Pull Request #6327