[HUDI-2687] [UMBRELLA] A new Trino connector for Hudi - ASF JIRA

Attach files

Attach Screenshot

Add vote

Voters

Watch issue

Watchers

Create sub-task

Link

Clone

Update Comment Author

Replace String in Comment

Update Comment Visibility

Delete Comments

XML

Word

Printable

JSON

Details

Type: Epic
Status: In Progress
Priority: Critical
Resolution: Unresolved
Affects Version/s: None
Fix Version/s: 0.14.0
Component/s: trino-presto
Labels:
- hudi-umbrellas

Story Points:
0
Epic Name:
Hudi Trino Connector

Description

This JIRA tracks all the tasks related to building a new Hudi connector in Trino.

Attachments

image-2021-11-05-14-16-57-324.png
05/Nov/21 06:16
0.3 kB
zhoukang
image-2021-11-05-14-17-03-211.png
05/Nov/21 06:17
0.3 kB
zhoukang

Issues in epic

quick-create-issue-for-epic-label

HUDI-2740	Support for snapshot querying on MOR table	Patch Available	Sagar Sumit	Actions
HUDI-3207	Hudi Trino connector PR review	Closed	Sagar Sumit	Actions
HUDI-4687	Avoid all illegal reflective access in the code	Closed	Sagar Sumit	Actions
HUDI-2265	Adding Trino to Docker setup	Closed	Unassigned	Actions
HUDI-2688	RFC 40 for Hudi Connector for Trino	Closed	Sagar Sumit	Actions
HUDI-2689	Support for snapshot query on COW table	Closed	Sagar Sumit	Actions
HUDI-2690	Support for read optimized queries on MOR table	Closed	Unassigned	Actions
HUDI-2691	Implement connector metadata	Closed	Sagar Sumit	Actions
HUDI-2692	Support for incremental queries on COW table	Open	Alberic	Actions
HUDI-2693	Support for incremental queries on MOR table	Open	Alberic	Actions
HUDI-2694	Support for ORC format	Open	Unassigned	Actions
HUDI-2721	Implement connector data sources	Closed	Sagar Sumit	Actions
HUDI-2722	Implement conenctor splits and split manager	Closed	Sagar Sumit	Actions
HUDI-2724	Benchmark connector	Closed	Ethan Guo	Actions
HUDI-2784	Add a hudi-trino-bundle	Closed	Ethan Guo	Actions
HUDI-2785	Create Trino setup in docker demo	Closed	Ethan Guo	Actions
HUDI-2979	Implement multiple splits for a single file	Closed	Ethan Guo	Actions
HUDI-2980	Support partitions for COW table	Closed	Ethan Guo	Actions
HUDI-3003	Support metadata-based listing	Closed	Sagar Sumit	Actions
HUDI-3004	Support bootstrap file splits	Closed	Ethan Guo	Actions
HUDI-3092	Implement partition filtering based on predicates in split manager/source	Closed	Ethan Guo	Actions
HUDI-3097	Address dependency issue with hudi-trino-bundle in connector	Closed	Sagar Sumit	Actions
HUDI-3126	Address whackamoles during testing of Hudi Trino connector	Closed	Ethan Guo	Actions
HUDI-3127	Add a new HoodieHFileReader for Trino with Java 11	Patch Available	Ethan Guo	Actions
HUDI-3153	Make Trino connector implementation extensible for different table/query types, data formats, etc.	Closed	Ethan Guo	Actions
HUDI-3196	Generalize partition key extraction when skipping Hive metastore	Closed	Ethan Guo	Actions
HUDI-4050	Upgrade Hudi version to 0.11.0 in the connector	Closed	Sagar Sumit	Actions
HUDI-3586	Add Trino IT tests following the docker demo	Closed	Ethan Guo	Actions
HUDI-3294	Verify file pruning	Closed	Sagar Sumit	Actions
HUDI-3296	Write tests using java write client and data generator	Closed	Unassigned	Actions
HUDI-3339	Reuse or implement caching like the hive connector	Open	Unassigned	Actions
HUDI-3630	Add Trino connector on top of v369	Closed	Sagar Sumit	Actions
HUDI-3816	Change HudiSplitSource to use async queue and shared executor service	Closed	Unassigned	Actions
HUDI-4789	Convert FileSystem usage in hudi connector to use TrinoFileSystem interface	Closed	Sagar Sumit	Actions
HUDI-4058	[trino-hudi connector] Support column stats pruning	Open	Unassigned	Actions
HUDI-4059	[trino-hudi connector] Support schema evolution	Open	Unassigned	Actions
HUDI-4594	Add integration test with MinIO	Closed	Sagar Sumit	Actions
HUDI-4595	Add connector documentation	Closed	Sagar Sumit	Actions
HUDI-4684	Support Glue catalog based metastore	Open	Unassigned	Actions
HUDI-4685	Create a separate HudiMetastore module	Open	Unassigned	Actions
HUDI-4788	Fix incorrect partition value for multi-partition table and add test	Closed	Sagar Sumit	Actions
HUDI-4956	Fix broken main method in Hudi query runners	Closed	Sagar Sumit	Actions
HUDI-4778	Upgrade to 0.12.1 Hudi version	Open	Unassigned	Actions
HUDI-5021	Build spark3 hudi image for trino product tests	Closed	Unassigned	Actions
HUDI-2723	Add product tests using spark3-hudi image	Closed	Sagar Sumit	Actions
HUDI-2695	[DOCS] Trino Hudi connector docs and update Hudi website	Closed	Sagar Sumit	Actions
HUDI-5115	Support for DDL/DML in Hudi connector	Open	Unassigned	Actions