[HUDI-7454] Flaky test: cdcStreaming{HoodieCDCSupplementalLoggingMode}[3]

• Type: Sub-task
• Status: Open
• Priority: Blocker
• Resolution: Unresolved
• Fix Version/s: 1.0.1
• Assignee: Danny Chen
• Reporter: Y Ethan Guo
• Created: 28/Feb/24 18:58
• Updated: 4 hours ago

Description

org.scalatest.exceptions.TestFailedException: 

Results do not match for query:
Timezone: sun.util.calendar.ZoneInfo[id="Etc/UTC",offset=0,dstSavings=0,useDaylight=false,transitions=0,lastRule=null]
Timezone Env: 

== Parsed Logical Plan ==
'Sort ['country ASC NULLS FIRST], true
+- Project [country#162815, population#162816L]
   +- Relation [_hoodie_commit_time#162810,_hoodie_commit_seqno#162811,_hoodie_record_key#162812,_hoodie_partition_path#162813,_hoodie_file_name#162814,country#162815,population#162816L,ts#162817] parquet

== Analyzed Logical Plan ==
country: string, population: bigint
Sort [country#162815 ASC NULLS FIRST], true
+- Project [country#162815, population#162816L]
   +- Relation [_hoodie_commit_time#162810,_hoodie_commit_seqno#162811,_hoodie_record_key#162812,_hoodie_partition_path#162813,_hoodie_file_name#162814,country#162815,population#162816L,ts#162817] parquet

== Optimized Logical Plan ==
Sort [country#162815 ASC NULLS FIRST], true
+- Project [country#162815, population#162816L]
   +- Relation [_hoodie_commit_time#162810,_hoodie_commit_seqno#162811,_hoodie_record_key#162812,_hoodie_partition_path#162813,_hoodie_file_name#162814,country#162815,population#162816L,ts#162817] parquet

== Physical Plan ==
AdaptiveSparkPlan isFinalPlan=true
+- == Final Plan ==
   *(1) Sort [country#162815 ASC NULLS FIRST], true, 0
   +- AQEShuffleRead coalesced
      +- ShuffleQueryStage 0
         +- Exchange rangepartitioning(country#162815 ASC NULLS FIRST, 4), ENSURE_REQUIREMENTS, [plan_id=71307]
            +- FileScan parquet [country#162815,population#162816L] Batched: false, DataFilters: [], Format: Parquet, Location: HoodieFileIndex(1 paths)[/tmp/junit8132202571330653449/dataset/country_to_population_table], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<country:string,population:bigint>
+- == Initial Plan ==
   Sort [country#162815 ASC NULLS FIRST], true, 0
   +- Exchange rangepartitioning(country#162815 ASC NULLS FIRST, 4), ENSURE_REQUIREMENTS, [plan_id=71307]
      +- FileScan parquet [country#162815,population#162816L] Batched: false, DataFilters: [], Format: Parquet, Location: HoodieFileIndex(1 paths)[/tmp/junit8132202571330653449/dataset/country_to_population_table], PartitionFilters: [], PushedFilters: [], ReadSchema: struct<country:string,population:bigint>

== Results ==

== Results ==
!== Correct Answer - 4 ==   == Spark Answer - 3 ==
!struct<>                   struct<country:string,population:bigint>
![Canada,1]                 [China,50]
![China,50]                 [Singapore,20]
![Singapore,22]             [US,205]
![US,204]                   
    
       
	at org.scalatest.Assertions.newAssertionFailedException(Assertions.scala:472)
	at org.scalatest.Assertions.newAssertionFailedException$(Assertions.scala:471)
	at org.apache.spark.sql.QueryTest$.newAssertionFailedException(QueryTest.scala:233)
	at org.scalatest.Assertions.fail(Assertions.scala:933)
	at org.scalatest.Assertions.fail$(Assertions.scala:929)
	at org.apache.spark.sql.QueryTest$.fail(QueryTest.scala:233)
	at org.apache.spark.sql.QueryTest$.checkAnswer(QueryTest.scala:243)
	at org.apache.hudi.functional.cdc.TestCDCStreamingSuite.cdcStreaming(TestCDCStreamingSuite.scala:207)
	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.lang.reflect.Method.invoke(Method.java:498)
	at org.junit.platform.commons.util.ReflectionUtils.invokeMethod(ReflectionUtils.java:688)
	at org.junit.jupiter.engine.execution.MethodInvocation.proceed(MethodInvocation.java:60)
	at org.junit.jupiter.engine.execution.InvocationInterceptorChain$ValidatingInvocation.proceed(InvocationInterceptorChain.java:131)
	at org.junit.jupiter.engine.extension.TimeoutExtension.intercept(TimeoutExtension.java:149)
	at org.junit.jupiter.engine.extension.TimeoutExtension.interceptTestableMet