Version 1.0.0-beta1

RELEASED

Start date not set

Released: 14/Nov/23

Release Notes

1–149 of 149View in Issue Navigator
PTKeySummaryAssigneeStatus
BlockerTaskHUDI-1046Support updates during clustering in CoW modeshenh062326Closed
BlockerImprovementHUDI-3187Initiate RFC Lock free concurrency control Vinoth ChandarClosed
BlockerTaskHUDI-3755Change the index plan to capture everything that is needed to create indexSagar SumitClosed
BlockerImprovementHUDI-4631Enhance retries for failed writes w/ write conflicts in a multi writer scenariosSagar SumitClosed
BlockerTaskHUDI-5148Write RFC for index functionSagar SumitClosed
BlockerNew FeatureHUDI-5249Support MetadataColumnStatsIndex for Spark recordLin LiuClosed
BlockerImprovementHUDI-6120fetchAllLogsMergedFileSlice will read basefile which it does not expectUnassignedClosed
BlockerNew FeatureHUDI-6654Add new log block header type to store record positionsEthan Guo (this is the old account; please use "yihua")Closed
BlockerNew FeatureHUDI-6702Extend merge API to support all merging operationsLin LiuClosed
BlockerNew FeatureHUDI-6785Introduce an engine-agnostic FileGroupReader for snapshot readEthan Guo (this is the old account; please use "yihua")Closed
BlockerNew FeatureHUDI-6786Integrate FileGroupReader with NewHoodieParquetFileFormat for Spark MOR Snapshot QueryLin LiuClosed
BlockerNew FeatureHUDI-6790Support incremental read in engine-agnostic FileGroupReaderLin LiuClosed
BlockerNew FeatureHUDI-6793Support time-travel read in engine-agnostic FileGroupReaderLin LiuClosed
BlockerNew FeatureHUDI-6795Implement generation of record_positions for updates and deletes on write pathSagar SumitClosed
BlockerNew FeatureHUDI-6796Implement position-based deletes in FileGroupReaderSagar SumitClosed
BlockerNew FeatureHUDI-6797Implement position-based updates in FileGroupReaderSagar SumitClosed
BlockerNew FeatureHUDI-6800Implement log writing with partial updates on the write pathEthan Guo (this is the old account; please use "yihua")Closed
BlockerNew FeatureHUDI-6801Implement merging of partial updates in FileGroupReaderEthan Guo (this is the old account; please use "yihua")Closed
BlockerTaskHUDI-6824Make sure serialization of log blocks is language and architecture independentEthan Guo (this is the old account; please use "yihua")Closed
BlockerTaskHUDI-6825Use UTF_8 to encode String to byte array in all placesEthan Guo (this is the old account; please use "yihua")Closed
BlockerTaskHUDI-6826Port BloomFilter related classes from Hadoop library to remove dependencyEthan Guo (this is the old account; please use "yihua")Closed
BlockerTaskHUDI-6850Add tests and docs for ported Bloom Filter classesEthan Guo (this is the old account; please use "yihua")Closed
BlockerImprovementHUDI-6874Move configs for reading a file group to hudi-common moduleEthan Guo (this is the old account; please use "yihua")Closed
BlockerBugHUDI-6946Data Duplicates with range pruning while using hoodie.bloom.index.use.metadataxi chaominClosed
BlockerBugHUDI-6956Fix CI failure on masterEthan Guo (this is the old account; please use "yihua")Closed
BlockerImprovementHUDI-6973Instantiate HoodieFileGroupRecordBuffer inside new file group readerEthan Guo (this is the old account; please use "yihua")Closed
BlockerImprovementHUDI-7076Turn on new features by default through configs for 1.0.0-beta1Ethan Guo (this is the old account; please use "yihua")Closed
CriticalImprovementHUDI-1623Solid completion time on timelineDanny ChenClosed
CriticalBugHUDI-5599Make sure that non-string primary-keys are supportedUnassignedClosed
CriticalBugHUDI-6873Clustering MOR applies base files after log filesJonathan VexlerClosed
CriticalBugHUDI-6892ExternalSpillableMap may cause data duplication when flink compactionUnassignedClosed
MajorNew FeatureHUDI-309General Redesign of Archived Timeline for efficient scan and managementDanny ChenClosed
MajorImprovementHUDI-2141Integration flink metric in flink streamZhaojing YuClosed
MajorImprovementHUDI-2503HoodieFlinkWriteClient supports to allow parallel writing to tables using Locking serviceNicholas JiangClosed
MajorTaskHUDI-3727Add metrics for async indexerSagar SumitClosed
MajorTaskHUDI-3756Clean up indexing APIs in write clientSagar SumitClosed
MajorImprovementHUDI-4756Clean up usages of "assume.date.partition" config within hudiLin LiuClosed
MajorTaskHUDI-5210End-to-end PoC of functional indexesSagar SumitClosed
MajorTaskHUDI-5211Add abstraction to track a function defined on a columnEthan Guo (this is the old account; please use "yihua")Closed
MajorTaskHUDI-5212Store index function in table propertiesEthan Guo (this is the old account; please use "yihua")Closed
MajorTaskHUDI-5213Support index function for Spark SQL built-in functions Ethan Guo (this is the old account; please use "yihua")Closed
MajorTaskHUDI-5214Add functionality to create new MT partition for index functionEthan Guo (this is the old account; please use "yihua")Closed
MajorTaskHUDI-5215Support file pruning based on new index function in SparkEthan Guo (this is the old account; please use "yihua")Closed
MajorTaskHUDI-5219Support "CREATE INDEX" for index function through Spark SQLEthan Guo (this is the old account; please use "yihua")Closed
MajorTaskHUDI-5672Archived Timeline as LSM Tree - Initial ImplDanny ChenClosed
MajorImprovementHUDI-6066HoodieTableSource supports parquet predicate push downNicholas JiangClosed
MajorImprovementHUDI-6336Support TimelineBased Checkpoint Metadata for flinkQijun FuClosed
MajorNew FeatureHUDI-6382support hudi table type changing in hudi-cli moduleKong WeiClosed
MajorNew FeatureHUDI-6480Flink lockless multi-writerDanny ChenClosed
MajorBugHUDI-6523Fix get valid checkpoint for current writerUnassignedClosed
MajorBugHUDI-6539New LSM tree style archived timelineDanny ChenClosed
MajorBugHUDI-6550Load Hadoop Conf into Hive Conf for Hive SyncUnassignedClosed
MajorImprovementHUDI-6641Remove the log append and always uses the current instant time in file nameDanny ChenClosed
MajorImprovementHUDI-6642Use completion time for file slicingDanny ChenClosed
MajorImprovementHUDI-6643Make the compaction non-serial (plan schedule and execution)Danny ChenClosed
MajorBugHUDI-6671Support 'alter table add partition' sqlUnassignedClosed
MajorImprovementHUDI-6676Add command for CreateHoodieTableLikeHui AnClosed
MajorTaskHUDI-6697Add doc for Flink Hudi CatalogUnassignedClosed
MajorImprovementHUDI-6701Explore use of UUID-6/7 as a replacement for current auto generated keysLin LiuClosed
MajorTaskHUDI-6708Support Record Index with the Async IndexerSagar SumitClosed
MajorTaskHUDI-6710Initial investigation to unearth gaps and unearth more open items.Sagar SumitClosed
MajorTaskHUDI-6711Write a RFC for Multi Table TxnsSagar SumitClosed
MajorNew FeatureHUDI-6720Prototype and benchmark position- and key-based updates and deletes in MOREthan Guo (this is the old account; please use "yihua")Closed
MajorNew FeatureHUDI-6721Prototype and benchmark partial updates in MOR log mergingEthan Guo (this is the old account; please use "yihua")Closed
MajorNew FeatureHUDI-6723Prototype and benchmark event-time based in MOR log mergingEthan Guo (this is the old account; please use "yihua")Closed
MajorImprovementHUDI-6725Support efficient completion time queries on the timelineDanny ChenClosed
MajorBugHUDI-6732Handle wildcards for partition paths passed in via spark-sqlvoonClosed
MajorImprovementHUDI-6742Remove the log file appending for multiple instantsDanny ChenClosed
MajorImprovementHUDI-6743Use current instant time as part of the log file nameDanny ChenClosed
MajorTaskHUDI-6751Scope out remaining work for the record merging APIEthan Guo (this is the old account; please use "yihua")Closed
MajorTaskHUDI-6752Scope out the work for file group reading and writing with record merging in SparkEthan Guo (this is the old account; please use "yihua")Closed
MajorBugHUDI-6766Fixing mysql debezium data lossUnassignedClosed
MajorNew FeatureHUDI-6770Improve on Key Lookup ReaderLin LiuClosed
MajorTaskHUDI-6775Add completion time in commit metadata filesDanny ChenClosed
MajorTaskHUDI-6776Unify commit metadata content in json for completed and avro for pending commitsSagar SumitClosed
MajorTaskHUDI-6779Audit current hoodie.propertiesSagar SumitClosed
MajorTaskHUDI-6780Replace classnames by modes/enums in table propertiesSagar SumitClosed
MajorTaskHUDI-6781Add deltacommit timestamp to log file nameDanny ChenClosed
MajorTaskHUDI-6782Instead of appending to same log file, consider one log file per commitDanny ChenClosed
MajorNew FeatureHUDI-6784Clean Merger API and its invocationsLin LiuClosed
MajorImprovementHUDI-6803Create marker file for every log fileUnassignedClosed
MajorImprovementHUDI-6805Print detailed error messages in clusteringUnassignedClosed
MajorImprovementHUDI-6809Optimizing the judgment of generating clustering plansUnassignedClosed
MajorNew FeatureHUDI-6810[RFC-46] Update merger API to support optional parametersLin LiuClosed
MajorImprovementHUDI-6818Create a database automatically when using the flink catalog dfs modeUnassignedClosed
MajorTaskHUDI-6821Make multiple base file formats within each file group.Sagar SumitClosed
MajorBugHUDI-6823writeTimer in emitCommitMetrics need to be initialized before usingUnassignedClosed
MajorBugHUDI-6827Exception would be thrown out if insert/insertOverWrite empty data into HUDI table using bulkInsertJing ZhangClosed
MajorBugHUDI-6828After Insert overwrite into a non-empty partition/table with empty datasets, the subsequent query results are incorrectJing ZhangClosed
MajorNew FeatureHUDI-6837Ensure the getInsertValue is wrapped correctlyUnassignedClosed
MajorTaskHUDI-6845Upgrade org.apache.pulsar:pulsar-client to 2.10.2UnassignedClosed
MajorBugHUDI-6846fix a bug of consistent bucket index clusteringUnassignedClosed
MajorTaskHUDI-6849Fix bootstrap procedure tests failure due to commit metadata parsingUnassignedClosed
MajorTaskHUDI-6859Fix ITTest clustering and compaction command testsUnassignedClosed
MajorBugHUDI-6862Replace directory connector markers in TestSqlStatementUnassignedClosed
MajorImprovementHUDI-6867Upgrade thrift's version to 0.13.0UnassignedClosed
MajorImprovementHUDI-6870[BigQuerySyncTool] Pass target project id when running job.UnassignedClosed
MajorTaskHUDI-6875Ensure record index covers file slices with no base filesSagar SumitClosed
MajorTestHUDI-6893Integration tests using docker demo do not test with latest hudi-trino-bundleSagar SumitClosed
MajorImprovementHUDI-6894ReflectionUtils is not thread safeUnassignedClosed
MajorImprovementHUDI-6900TestInsertTable "Test Bulk Insert Into Consistent Hashing Bucket Index Table" is failing continuouslyUnassignedClosed
MajorTestHUDI-6907E2E support HoodieSparkRecord Lin LiuClosed
MajorSub-taskHUDI-6908Verify if any gaps exists for the e2e supportLin LiuClosed
MajorNew FeatureHUDI-6919Support Partial Merging in HoodieFileGroupReaderLin LiuClosed
MajorBugHUDI-6922Fix inconsistency between base file format and catalog input formatUnassignedClosed
MajorBugHUDI-6924Fix hoodie table config not wok in table propertiesWecharClosed
MajorBugHUDI-6926Disable DROP_PARTITION_COLUMNS when upsert MOR tableUnassignedClosed
MajorNew FeatureHUDI-6928Support position based merging in HoodieFileGroupReaderLin LiuClosed
MajorImprovementHUDI-6929Lazy loading dynamically for CompletionTimeQueryViewDanny ChenClosed
MajorImprovementHUDI-6937CopyOnWriteInsertHandler#consume will cause clustering performance degradationUnassignedClosed
MajorSub-taskHUDI-6938Run TPC-DS benchmark on the integrationLin LiuClosed
MajorImprovementHUDI-6939Add async archiving for FlinkUnassignedClosed
MajorBugHUDI-6944Fix Flink bootstrap concurrency issue causes job to keep rolling backYue ZhangClosed
MajorBugHUDI-6945Fix HoodieRowDataParquetWriter cast issueYue ZhangClosed
MajorImprovementHUDI-6952Skip reading the uncommitted log files for log readerDanny ChenClosed
MajorBugHUDI-6959Do not rollback current instant when bulk insert as row failedUnassignedClosed
MajorNew FeatureHUDI-6962Correct the behavior of bulk insert for NB-CC Jing ZhangClosed
MajorImprovementHUDI-6965Flink Quickstart RestructuringUnassignedClosed
MajorImprovementHUDI-6975Optimize the implementation of DayBasedCompactionStrategyUnassignedClosed
MajorTaskHUDI-6982Run LST benchmark and Collect performance statsLin LiuClosed
MajorImprovementHUDI-6990Configurable clustering task parallelismUnassignedClosed
MajorBugHUDI-6991Fix hoodie.parquet.max.file.size conf reset errorUnassignedClosed
MajorNew FeatureHUDI-6993Support Flink 1.18UnassignedClosed
MajorNew FeatureHUDI-6997Create a new WriteConcurrencyMode type for non-blocking concurrency controlJing ZhangClosed
MajorBugHUDI-6998Fix drop table failure when load table as spark v2 table whose path is deleteWecharClosed
MajorBugHUDI-7001Fix to keep the _hoodie_record_key value format of bulk_insert and upsert consistentUnassignedClosed
MajorBugHUDI-7005Flink SQL Queries on Hudi Table fail when using the hudi-aws-bundle jarUnassignedClosed
MajorImprovementHUDI-7012The BootstrapOperator reduces the memory.Bo CuiClosed
MajorBugHUDI-7013Drop table command cannot delete dir when purge is enable xyClosed
MajorBugHUDI-7017Prevent full schema evolution from wrongly falling back to OOBvoonClosed
MajorImprovementHUDI-7019Add instant details consumer to HoodieArchivedTimelineUnassignedClosed
MajorImprovementHUDI-7022RunClusteringProcedure support limit parameterUnassignedClosed
MajorBugHUDI-7030Log reader data lost as that not consistent behavior in timeline's containsInstant UnassignedClosed
MajorImprovementHUDI-7038RunCompactionProcedure support limit parameterUnassignedClosed
MajorImprovementHUDI-7039PartialUpdateAvroPayload preCombine failed need show detailsxyClosed
MajorBugHUDI-7042Fix filegroup readerUnassignedClosed
MajorImprovementHUDI-7048Fix checkpoint loss issue when changing MOR to COW in streamerUnassignedClosed
MajorImprovementHUDI-7057Support CopyToTableProcedure with patitial column copy xyClosed
MajorTaskHUDI-7092Release notes for 1.0.0-beta1Sagar SumitClosed
MinorBugHUDI-3786how to deduce what MDT partitions to update on the write path w/ async indeingSagar SumitClosed
MinorImprovementHUDI-6142Refactor the code related to creating user-defined indexUnassignedClosed
MinorImprovementHUDI-6581Remove unnecessary validations in function getOldestInstantToRetainForClusteringUnassignedClosed
MinorImprovementHUDI-6739Avoid checking timeline for successful commits for spark structured streaming when offset is 0UnassignedClosed
MinorImprovementHUDI-6773Add Test cases to show case insert into behaviour with different mergers and payload.UnassignedClosed
MinorImprovementHUDI-6853ArchiveCommitsProcedure should throw an exception when the archive operation executes failedUnassignedClosed
MinorImprovementHUDI-6969Add speed limit for stream readUnassignedClosed
MinorImprovementHUDI-6992IncrementalInputSplits incorrectly set the latestCommit attrUnassignedClosed
MinorImprovementHUDI-7011 a metric to indicate whether rollback has occurred in final compaction state UnassignedClosed
TrivialImprovementHUDI-6968remove block logical in BulkInsertWriteFunction#openUnassignedClosed
1–149 of 149