[AVRO-1980] Write to Avro File in Bulk

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Affects Version/s: 1.8.1
• Component/s: java
• Assignee: Unassigned
• Reporter: Santosh Balasubramanya
• Created: 09/Jan/17 14:15
• Updated: 13/Oct/19 01:46

Description

when writing to Avro files usually append happens record by record.
Can't it be done by buffering and then committing it to file?
Below example
DatumWriter<User> userDatumWriter = new SpecificDatumWriter<User>(User.class);
DataFileWriter<User> dataFileWriter = new DataFileWriter<User>(userDatumWriter);
dataFileWriter.create(user1.getSchema(), new File("users.avro"));
dataFileWriter.append(user1);
dataFileWriter.append(user2);
dataFileWriter.append(user3);
dataFileWriter.close();