Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
Description
TestBootstrapRead.java In the attached file, enable the timeline serverĀ
'hoodie.embed.timeline.server'. It will occasionally fail in metadata or mixed mode because some records will be null besides the metadata columns:
+-------------------+---------------------+------------------------------------+-------------------------+------------------------------------------------------------------------+------------------+------------------------------------+------------------+-------------------+-------------+------------+----------+------------------+----------+-------------------+-------------------+------------------------+-------------------+--------+----------+--------------+---------+--------------------+---------+--------------------------+----------+|_hoodie_commit_time|_hoodie_commit_seqno |_hoodie_record_key |_hoodie_partition_path |_hoodie_file_name |_hoodie_is_deleted|_row_key |begin_lat |begin_lon |city_to_state|current_date|current_ts|distance_in_meters|driver |end_lat |end_lon |fare |height |nation |partition |partition_path|rider |seconds_since_epoch |timestamp|tip_history |weight |+-------------------+---------------------+------------------------------------+-------------------------+------------------------------------------------------------------------+------------------+------------------------------------+------------------+-------------------+-------------+------------+----------+------------------+----------+-------------------+-------------------+------------------------+-------------------+--------+----------+--------------+---------+--------------------+---------+--------------------------+----------+|00000000000001 |00000000000001_4_0 |876743b0-f5e7-4289-b13b-1a0404d94380|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null ||00000000000001 |00000000000001_4_1 |00923d1a-58fc-4d42-8953-4a47b47d738f|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null ||20230510125841762 |20230510125841762_1_2|b318c482-8e43-4614-bdab-80946d5a9f53|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|false |b318c482-8e43-4614-bdab-80946d5a9f53|0.5285807377766387|0.12835359814395741|[CA] |12 |1047178778|521899450 |driver-001|0.41394620067559684|0.08532822423986208|[42.25978252084417, USD]|[0, 0, 7, -91, -36]|[Canada]|2015-03-16|2015-03-16 |rider-001|-2845295541651788027|0 |[[32.10533813167099, USD]]|0.59076524||00000000000001 |00000000000001_4_3 |4dcc72d7-0878-41c2-a85d-3e6374b88bb8|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null ||00000000000001 |00000000000001_4_4 |cfa79530-fc9f-42de-a181-34c06e79d9c5|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null ||00000000000001 |00000000000001_4_5 |4d2a4755-83a7-4201-9b65-0148752d55b7|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null ||00000000000001 |00000000000001_4_6 |76429084-f78e-4c6d-a70d-088cb5d955aa|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null ||00000000000001 |00000000000001_4_7 |8436456b-9858-45cb-8a07-bdae536a2d17|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null ||00000000000001 |00000000000001_4_8 |b6ca00ed-a4f4-4db5-a269-6be250b64caa|partition_path=2015-03-16|7e1dea56-e88c-4072-be61-f4ae01feaaa3_1-138-381_20230510125841762.parquet|null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |null |+-------------------+---------------------+------------------------------------+-------------------------+------------------------------------------------------------------------+------------------+------------------------------------+------------------+-------------------+-------------+------------+----------+------------------+----------+-------------------+-------------------+------------------------+-------------------+--------+----------+--------------+---------+--------------------+---------+--------------------------+----------++-------------------+--------------------+------------------------------------+-------------------------+-------------------------------------------------------------------+
This is due to the bootstrap base file not being set, so HoodieMergeHelper does not process the file as a bootstrap skeleton file
Attachments
Attachments
Issue Links
- relates to
-
HUDI-6249 Make maps in HoodieTableFileSystemView concurrent maps
- Reopened
- links to