Step 1 of 4: Choose Issues

Cancel

T Patch Info Key Summary Assignee Reporter P Status Resolution Created Updated Due Development
Sub-task SPARK-32935

SPARK-27589 File source V2: support bucketing

Unassigned Gengliang Wang Major Open Unresolved  
Sub-task SPARK-31560

SPARK-27589 Add V1/V2 tests for TextSuite and WholeTextFileSuite

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-30628

SPARK-27589 File source V2: Support partition pruning with subqueries

Unassigned Gengliang Wang Major Open Unresolved  
Sub-task SPARK-30627

SPARK-27589 Disable all the V2 file sources in Spark 3.0 by default

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-30428

SPARK-27589 File source V2: support partition pruning

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-28396

SPARK-27589 Add PathCatalog for data source V2

Unassigned Gengliang Wang Major Resolved Won't Fix  
Sub-task SPARK-28218

SPARK-27589 Migrate Avro to File source V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-28205

SPARK-27589 useV1SourceList configuration should be for all data sources

Gengliang Wang Gengliang Wang Minor Resolved Fixed  
Sub-task SPARK-28089

SPARK-27589 File source v2: support reading output of file streaming Sink

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27926

SPARK-27589 Allow altering table add columns with CSVFileFormat/JsonFileFormat provider

Gengliang Wang Gengliang Wang Minor Resolved Fixed  
Sub-task SPARK-27849

SPARK-27589 Redact treeString of FileTable and DataSourceV2ScanExecBase

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27668

SPARK-27589 File source V2: support reporting statistics

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27580

SPARK-27589 Implement `doCanonicalize` in BatchScanExec for comparing query plan results

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27504

SPARK-27589 File source V2: support refreshing metadata cache

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27490

SPARK-27589 File source V2: return correct result for Dataset.inputFiles()

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27459

SPARK-27589 Revise the exception message of schema inference failure in file source V2

Gengliang Wang Gengliang Wang Trivial Resolved Fixed  
Sub-task SPARK-27448

SPARK-27589 File source V2 table provider should be compatible with V1 provider

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27443

SPARK-27589 Support UDF input_file_name in file source V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27435

SPARK-27589 Support schema pruning in Orc V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27418

SPARK-27589 Migrate Parquet to File Data Source V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27407

SPARK-27589 File source V2: Invalidate cache data on overwrite/append

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27384

SPARK-27589 File source V2: Prune unnecessary partition columns

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27356

SPARK-27589 File source V2: return actual schema in method `FileScan.readSchema`

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27326

SPARK-27589 Fall back all v2 file sources in `InsertIntoTable` to V1 FileFormat

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27291

SPARK-27589 File source V2: Ignore empty files in load

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27286

SPARK-27589 Handles exceptions on proceeding to next record in FilePartitionReader

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27271

SPARK-27589 Migrate Text to File Data Source V2

Unassigned Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27269

SPARK-27589 File source v2 should validate data schema only

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27136

SPARK-27589 Remove data source option check_files_exist

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27132

SPARK-27589 Improve file source V2 framework

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27128

SPARK-27589 Migrate JSON to File Data Source V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27085

SPARK-27589 Migrate CSV to File Data Source V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-27049

SPARK-27589 Support handling partition values in the abstraction of file source V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-26871

SPARK-27589 File Source V2: avoid creating unnecessary FileIndex in the write path

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-26744

SPARK-27589 Support schema validation in File Source V2

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-26673

SPARK-27589 File source V2 write: create framework and migrate ORC to it

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-26447

SPARK-27589 Allow OrcColumnarBatchReader to return less partition columns

Gengliang Wang Gengliang Wang Major Resolved Fixed  
Sub-task SPARK-23817

SPARK-27589 Create file source V2 framework and migrate ORC read path

Gengliang Wang Gengliang Wang Major Resolved Fixed  

Cancel