Description
SPARK_PARTITION_ID column should return the partition index of the Spark partition. Note that we already have a DataFrame function for it: https://github.com/apache/spark/blob/78a6723e8758b429f877166973cc4f1bbfce73c4/sql/core/src/main/scala/org/apache/spark/sql/functions.scala#L705
Attachments
Issue Links
- links to