Major Description
Currently the storage-partitioned join requires both sides to be fully clustered on the partition values, that is, all input partitions reported by a V2 data source shall be grouped by partition values before the join happens. This could lead to data skew issues if a particular partition value is associated with a large amount of rows.
To combat this, we can introduce the idea of partially clustered distribution, which means that only one side of the join is required to be fully clustered, while the other side is not. This allows Spark to increase the parallelism of the join and avoid the data skewness.
