[HUDI-1574] Trim existing unit tests to finish in much shorter amount of time

• Type: Epic
• Status: Open
• Priority: Critical
• Resolution: Unresolved
• Affects Version/s: 0.9.0
• Fix Version/s: 1.1.0, 0.15.0
• Component/s: Testing, tests-ci
• Story Points: 0
• Epic Name: Trim down UT runtime
• Assignee: Unassigned
• Reporter: Vinoth Chandar
• Created: 02/Feb/21 22:04
• Updated: 15/Aug/23 00:51

Description

spark-client-tests

278.165 s - in org.apache.hudi.table.TestHoodieMergeOnReadTable
201.628 s - in org.apache.hudi.metadata.TestHoodieBackedMetadata
185.716 s - in org.apache.hudi.client.TestHoodieClientOnCopyOnWriteStorage
158.361 s - in org.apache.hudi.index.TestHoodieIndex
156.196 s - in org.apache.hudi.table.TestCleaner
132.369 s - in org.apache.hudi.table.action.commit.TestCopyOnWriteActionExecutor
93.307 s - in org.apache.hudi.table.action.compact.TestAsyncCompaction
67.301 s - in org.apache.hudi.table.upgrade.TestUpgradeDowngrade
45.794 s - in org.apache.hudi.client.TestHoodieReadClient
38.615 s - in org.apache.hudi.index.bloom.TestHoodieBloomIndex
31.181 s - in org.apache.hudi.client.TestTableSchemaEvolution
20.072 s - in org.apache.hudi.table.action.compact.TestInlineCompaction

grep " Time elapsed" hudi-client/hudi-spark-client/target/surefire-reports/* | awk -F',' '

{ print $5 }

' | awk -F':' '

{ print $2 }

' | sort -nr | less

hudi-utilities

209.936 s - in org.apache.hudi.utilities.functional.TestHoodieDeltaStreamer
204.653 s - in org.apache.hudi.utilities.functional.TestHoodieMultiTableDeltaStreamer
34.116 s - in org.apache.hudi.utilities.sources.TestKafkaSource
29.865 s - in org.apache.hudi.utilities.sources.TestParquetDFSSource
26.189 s - in org.apache.hudi.utilities.sources.helpers.TestDatePartitionPathSelector

Other Tests

42.595 s - in org.apache.hudi.common.functional.TestHoodieLogFormat
38.918 s - in org.apache.hudi.common.bootstrap.TestBootstrapIndex
22.046 s - in org.apache.hudi.common.functional.TestHoodieLogFormatAppendFailure