Export - CSV (All fields)
Export - CSV (Current fields)
SPARK-31934
Don't print out extra info from the docker image tool if passed --helpSPARK-33831
Update Jetty to 9.4.34
Column.__repr__ shouldn't encode JVM response
Make some keywords reserved along with the ANSI/SQL standard
Adding support for core writersSPARK-30010
Remove deprecated SparkConf.setAll(Traversable)
Update Kafka Headers Documentation in Structured Streaming
SPARK-31239
Fix flaky test: WorkerDecommissionSuite.verify a task with all workers decommissioned succeeds
Exit the executor once all tasks & migrations are finished
Add trait to easily whitelist logical operators that produce named output from CleanupAliases
Improve HistoryServerDiskManager.updateAccessTime()
Make Spark's guava version configurable from the maven command line.
NullPointerException when parsing event log with InMemoryStore
Reading parquet with Decimal(8,2) written as a Decimal64 blows up
HiveExternalCatalogVersionsSuite shouldn't use or delete hard-coded /tmp directory
YARN: Print direct links to driver logs alongside application report in cluster modeSPARK-33166
Provide Search Function in Spark docs site
Built-in SQL Function Support - WIDTH_BUCKET
Remove references to org.spark-project.hive
Add more constraint checks in file streaming source to avoid checkpoint corruption
Image data source supports sample pushdown
adaptively blockify instances
Upgrade to scalatest 3.2.0 for Scala 2.13.3 compatibilitySPARK-29909
Drop Python 2 and Python 3.4 and 3.5.
Kubernetes GA Preparation
IllegalArgumentException when applying RemoveRedundantSorts before EnsureRequirements
Support PYSPARK_PYTHON and PYSPARK_DRIVER_PYTHON environment variablesSPARK-32981
Remove hive-1.2/hadoop-2.7 from Apache Spark 3.1 distribution
Use Apache Hadoop 3.2.0 dependency by default
Dataframe add sameResult and sementicHash methodSPARK-30762
Add dtype="float32" support to vector_to_array UDF
Support local dependency uploading from --py-files
new write APIs for data source v2
put all the backward compatible behavior change configs under spark.sql.legacy.*
Informational Referential Integrity Constraints Support in Spark
Copy cache data when node is being shut down
setDelegateCatalog should be called if catalog has implemented CatalogExtension
Support simple all gather in barrier task context
Update Docs Bootstrap to 4.4.1
Make Pyspark Hadoop 3.2+ Variant available in PyPI
Drop references to slave
Update SQL ALTER TABLE RENAME to use TableCatalog API- SPARK-28704
Test backward compatibility on JDK9+ once we have a version supports JDK9+
Capture the missing rules in HiveSessionStateBuilder
Don't fail running jobs when decommissioned executors finally go away
support Dynamic Partition Pruning in Adaptive Execution
Add metadata around semi-structured columns to Spark
Simplify API for JSON access in DataFrames/SQL
Unify create table syntax to fix ambiguous two different CREATE TABLE syntaxes
Support of map, filter, withField, dropFields in nested list of structures