[AVRO-2166] Wrong Sync header appended to the file.

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Not A Bug
• Affects Version/s: 1.8.2
• Component/s: java
• Release Note: This was an internal coding issue. Apologize for the same.
• Assignee: Unassigned
• Reporter: mukesh katariya
• Created: 04/Apr/18 16:29
• Updated: 24/Apr/18 06:32
• Resolved: 24/Apr/18 06:32

Description

We are using the following code, pseudo code for representation.
 

 File file1 = new File("/tmp/tmpdata/message-" + System.currentTimeMillis());
 FileOutputStream out = new FileOutputStream(file1, false);
 DatumWriter datumWriter = new GenericDatumWriter<>();
 writer = new DataFileWriter(datumWriter);
 writer.setCodec(CodecFactory.snappyCodec());
 writer.create(avroSchema, out);
 
//Records are from sink connector of kafka.   
//Iterator over the records and bundling them together.
 for( set of records ... )
    writer.append(value);
 writer.close();

The issue which we are facing is the sync code is not matching.

 Original file with sync code at the end is wrong. avro tool failed with error invalid sync.

Since the header.sync and sync bytes after the block didnot match.

manually -replaced the 16 byte with sync bytes from the header section. the file i was able to read using avro tools.

 Thanks and Best Regards

Mukesh