[HUDI-784] CorruptedLogFileException sometimes happens on GCS

• Type: Bug
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.6.0
• Labels: pull-request-available
• Assignee: Unassigned Assign to me
• Reporter: Alexander Filipchik
• Created: 11/Apr/20 21:34
• Updated: 10/May/20 10:01
• Resolved: 10/May/20 10:01

Description

768726 [Executor task launch worker-2] ERROR org.apache.hudi.common.table.log.AbstractHoodieLogRecordScanner - Got exception when reading log file
org.apache.hudi.exception.CorruptedLogFileException: HoodieLogFile

{pathStr=' [gs://.log.|gs://1_20200219014757.log.2] ', fileLen=0}

could not be read. Did not find the magic bytes at the start of the block
at org.apache.hudi.common.table.log.HoodieLogFileReader.readMagic(HoodieLogFileReader.java:313)
at org.apache.hudi.common.table.log.HoodieLogFileReader.hasNext(HoodieLogFileReader.java:295)
at org.apache.hudi.common.table.log.HoodieLogFormatReader.hasNext(HoodieLogFormatReader.java:103)
 

I did extensive debugging and still unclear on why it is happening. It might be issue with GCS libraries themselves. The fix that is working:

 

In: HoodieLogFileReader made 

// private final byte[] magicBuffer = new byte[6];

non static. I'm not sure why it is actually static in the first place as it is inviting a race.

Also in HoodieLogFileReader:

added

// fsDataInputStream.seek(0);

added right after stream creation in the constructor.

 

 

 

 

 

 

Issue Links

• links to: GitHub Pull Request #1516