Export - CSV (All fields)
Export - CSV (Current fields)
SPARK-38069
improve structured streaming window of calculatedSPARK-35746
Task id in the Stage page timeline is incorrect
Improve error message in default spark session catalog
Use WeakReference over SoftReference in LevelDB to avoid retaining iterator refs
Update API usage on pyspark pandas documents
Pin jinja2 in spark-rm/Dockerfile and add as a required dependency in the release README.md
Add support for calling debugCodegen from Python & Java
Duplicate code implementation of `WritablePartitionedIterator`
Move the old partition data to trash instead of deleting it when inserting rewrite hive table
Add trait to easily whitelist logical operators that produce named output from CleanupAliasesSPARK-19256
Hive bucketing write support
Spark aggregate improvement
Map JDBC SQL TIME type to TimestampType with time portion fixed regardless of timezone
Lost the null flag info when show create table
add fs.s3a.endpoint if unset and fs.s3a.endpoint.region is null
Support user defined initial state with flatMapGroupsWithState in Structured StreamingSPARK-35800
Improving testability of GroupState in streaming flatMapGroupsWithState
aditya
Spark fails to launch executors with very large user classpath lists on YARNSPARK-35670
Upgrade ZSTD-JNI to 1.5.0-2
Enable push-based shuffle when multiple app attempts are enabled and manage concurrent access to the state in a better way
Upgrade kubernetes-client to 4.13.2
Spark artefacts built against Scala 2.13 incorrectly depend on Scala 2.12
Disable validate default values when parsing Avro schemas
Explicit representation of star in MergeIntoTable's Update and Insert action
Port/integrate Koalas documentation into PySpark
Incorrect star expansion logic MERGE INSERT * / UPDATE *
Support traversal pruning in the transform function family
MergeIntoTable should avoid ambiguous reference
Add RocksDB StateStore implementation
Executors don't get rescheduled in standalone mode when worker dies
Completely fix the broadcast timeout issue in AQE
Upgrade ORC to 1.6.6
Improve ZSTD support
BoundAttribute issue after optimization by BooleanSimplification and PushFoldableIntoBranches
Propagating SPARK_CONF_DIR in K8s and tests
Stabilize Data Source V2 API
Image data source supports sample pushdown
Upgrade to Hadoop 3.3.1
Upgrade Kafka to 2.8.0
JDBC V2 Datasource aggregate push downSPARK-30978
Remove multiple workers on the same host support from Standalone backend
use MemoryUsage to control the size of block
Correlated scalar subqueries must be aggregated
Build and test Spark against Scala 2.13
Capture the missing rules in HiveSessionStateBuilder
New type coercion syntax rules in ANSI mode
Support configurable Avro schema field matching for positional or by-name
Add metadata around semi-structured columns to Spark
Simplify API for JSON access in DataFrames/SQL