[HUDI-7768] Fix failing tests for 0.15.0 release (async compaction and metadata num commits check)

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.15.0
• Component/s: tests-ci
• Labels: pull-request-available
• Assignee: sivabalan narayanan
• Reporter: sivabalan narayanan
• Created: 15/May/24 21:04
• Updated: 11/Jun/24 01:13
• Resolved: 11/Jun/24 01:13

Description

 

 

https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=23953&view=logs&j=600e7de6-e133-5e69-e615-50ee129b3c08&t=bbbd7bcc-ae73-56b8-887a-cd2d6deaafc7 https://dev.azure.com/apache-hudi-ci-org/apache-hudi-ci/_build/results?buildId=23953&view=logs&j=7601efb9-4019-552e-11ba-eb31b66593b2&t=d4b4e11d-8e26-50e5-a0d9-bb2d5decfeb9 org.apache.hudi.exception.HoodieMetadataException: Metadata table's deltacommits exceeded 3: this is likely caused by a pending instant in the data table. Resolve the pending instant or adjust `hoodie.metadata.max.deltacommits.when_pending`, then restart the pipeline. at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.checkNumDeltaCommits(HoodieBackedTableMetadataWriter.java:835) at org.apache.hudi.metadata.HoodieBackedTableMetadataWriter.validateTimelineBeforeSchedulingCompaction(HoodieBackedTableMetadataWriter.java:1367)

java.lang.IllegalArgumentException: Following instants have timestamps >= compactionInstant (002) Instants :[[004__deltacommit__COMPLETED__20240515123806398]] at org.apache.hudi.common.util.ValidationUtils.checkArgument(ValidationUtils.java:42) at org.apache.hudi.table.action.compact.ScheduleCompactionActionExecutor.execute(ScheduleCompactionActionExecutor.java:108)

Issue Links

• links to: GitHub Pull Request #11232