[AVRO-1873] avro gem doesn't compatible with other languages with snappy compression

• Type: Bug
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 1.8.1
• Fix Version/s: 1.8.2
• Component/s: ruby

Environment

CentOS 6.8 64bit, Snappy 1.1.0, Python 3.5, Ruby 2.2.3

• Assignee: Ryan Blue
• Reporter: Pumsuk Cho
• Created: 30/Jun/16 08:15
• Updated: 12/Sep/16 16:09
• Resolved: 12/Sep/16 16:09

Description

I've tested avro gem today, then found some weird result.

With python library like "fastavro", generated an avro file snappy compressed. This file works fine with avro-tools-1.8.1.jar.

java -jar avro-tools-1.8.1.jar tojson testing.avro returns what I expected.

But NOT compatible with ruby using avro gem returns "Invalid Input" message. And snappy compressed avro file made with avro gem doesn't work with avro-tools nor in python with avro-python3 and fastavro.

my ruby codes are below:

schema = Avro::Schema.paese(File.open('test.avsc', 'r').read)
avrofile = File.open('test.avro', 'wb')
writer = Avro::IO::DatumWriter.new(schema)
datawriter = Avro::DataFile::Writer.new file, writer, schema, 'snappy'
datawriter<<

{"title" => "Avro", "author" => "Apache Foundation"}

datawriter.close

Issue Links

• blocks: AVRO-1885 Release 1.8.2
• links to: GitHub Pull Request #121