Uploaded image for project: 'Pig'
  1. Pig
  2. PIG-2201

AvroStorage bug in testRecordWithSplit() unit test

    Details

    • Type: Bug
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: None
    • Labels:
      None

      Description

      The AvroStorage.testRecordWithSplit() unit test seems to be reporting a false positive to me. The test basically reads from a set of data and groups on a field memberId and prints counts of total occurrences. Using the Avro tool to inspect the contents of the source data versus the expected data it appears a count is off. Note the 8 and the 7 below.

      $ java org.apache.avro.tool.Main tojson \
        java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/test_record.avro \
        | grep -c '"member_id":1244'
      8
      
      $ java org.apache.avro.tool.Main tojson \
        java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/expected_testRecordSplit1.avro 
      {"member_id":1211,"count":3}
      {"member_id":1214,"count":1}
      {"member_id":1221,"count":1}
      {"member_id":1244,"count":7}
      {"member_id":1246,"count":1}
      {"member_id":1253,"count":1}
      {"member_id":1256,"count":3}
      {"member_id":1257,"count":2}
      
      $ java org.apache.avro.tool.Main tojson \
        java/src/test/java/org/apache/pig/piggybank/test/storage/avro/avro_test_files/expected_testRecordSplit2.avro 
      {"key":{"int":1211},"cnt":{"long":3}}
      {"key":{"int":1214},"cnt":{"long":1}}
      {"key":{"int":1221},"cnt":{"long":1}}
      {"key":{"int":1244},"cnt":{"long":7}}
      {"key":{"int":1246},"cnt":{"long":1}}
      {"key":{"int":1253},"cnt":{"long":1}}
      {"key":{"int":1256},"cnt":{"long":3}}
      {"key":{"int":1257},"cnt":{"long":2}}
      

        Activity

        There are no comments yet on this issue.

          People

          • Assignee:
            billgraham Bill Graham
            Reporter:
            billgraham Bill Graham
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:

              Development