Details
-
Bug
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
The AvroStorage.testRecordWithSplit() unit test seems to be reporting a false positive to me. The test basically reads from a set of data and groups on a field memberId and prints counts of total occurrences. Using the Avro tool to inspect the contents of the source data versus the expected data it appears a count is off. Note the 8 and the 7 below.
$ java org.apache.avro.tool.Main tojson \
java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_record.avro \
| grep -c '"member_id":1244'
8
$ java org.apache.avro.tool.Main tojson \
java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/expected_testRecordSplit1.avro
{"member_id":1211,"count":3}
{"member_id":1214,"count":1}
{"member_id":1221,"count":1}
{"member_id":1244,"count":7}
{"member_id":1246,"count":1}
{"member_id":1253,"count":1}
{"member_id":1256,"count":3}
{"member_id":1257,"count":2}
$ java org.apache.avro.tool.Main tojson \
java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/expected_testRecordSplit2.avro
{"key":{"int":1211},"cnt":{"long":3}}
{"key":{"int":1214},"cnt":{"long":1}}
{"key":{"int":1221},"cnt":{"long":1}}
{"key":{"int":1244},"cnt":{"long":7}}
{"key":{"int":1246},"cnt":{"long":1}}
{"key":{"int":1253},"cnt":{"long":1}}
{"key":{"int":1256},"cnt":{"long":3}}
{"key":{"int":1257},"cnt":{"long":2}}