[HUDI-1514] Avoid raw type use for parameter of interface

• Type: Improvement
• Status: Closed
• Priority: Minor
• Resolution: Fixed
• Fix Version/s: 0.7.0
• Component/s: Utilities
• Labels: pull-request-available
• Assignee: puyvqi
• Reporter: puyvqi
• Created: 07/Jan/21 09:48
• Updated: 17/Nov/24 10:03
• Resolved: 09/Jan/21 10:39

Description

org.apache.hudi.utilities.transform.Transformer  method api:

Dataset apply(JavaSparkContext jsc, SparkSession sparkSession, Dataset<Row> rowDataset, TypedProperties properties);

it's a bad idea to use raw type parameter(without a type parameter or wildcard) in interface ,especially Dataset[T] is define in scala,when implementing this interface in scala. it turn out to be an compilation error:

//overriding method apply in trait Transformer of type (x$1: org.apache.spark.api.java.JavaSparkContext, x$2: org.apache.spark.sql.SparkSession, x$3: org.apache.spark.sql.Dataset[org.apache.spark.sql.Row], x$4: org.apache.hudi.common.config.TypedProperties)org.apache.spark.sql.Dataset;

method apply has incompatible type
  override def apply(jsc: JavaSparkContext, sparkSession: SparkSession, rowDataset: Dataset[Row], properties: TypedProperties): Dataset[_] = rowDataset

//method apply has incompatible type
  override def apply(jsc: JavaSparkContext, sparkSession: SparkSession, rowDataset: Dataset[Row], properties: TypedProperties): Dataset[Row] = rowDataset

Issue Links

• links to: GitHub Pull Request #2420