Avro
  1. Avro
  2. AVRO-313

Default values for fields or records and array (or map) don't work with ResolvingDecoder

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.3.0
    • Component/s: java
    • Labels:
      None

      Description

      If the writer's and reader's schema are different and reader's schema defines a field with a default while the writer's does not have that field, the ResolovingDecoder returns the default value for the field to the reader. This is accomplished by replacing the actual InputStream with the one that has the default value. Remember the actual input stream didn't have a value for the field. Now the original input stream is restored by an implicit action called DEFAULT_END_ACTION. If the record is the last in a slice of an array(or a map), the code used to see (after consuming the record) if there are any more entries in the array(map). This check should happen on the original input stream. Since the input stream has not been restored, the check happens on the default-value input stream causing error.

      The following patch solves this by consuming all "trailing" implicit actions before looking for further entries in the array(map). There was already a method Parser.processImplicitActions() which consumed all implicit actions. It turns out one needs to consume only the trailing implicit action. So I renamed the method to processTrailingImplicitActions and changed the code to meet the new specification.

      I also added a couple of tests that catch the problem.

      First apply the patch for the tests, you'd see the problem. Then apply the fix and see that the problem go away.

      1. AVRO-313-test.patch
        3 kB
        Thiruvalluvan M. G.
      2. AVRO-313.patch
        6 kB
        Thiruvalluvan M. G.

        Activity

        Hide
        Doug Cutting added a comment -

        +1 This looks good to me.

        Show
        Doug Cutting added a comment - +1 This looks good to me.
        Hide
        Thiruvalluvan M. G. added a comment -

        Committed revision 899185.

        Show
        Thiruvalluvan M. G. added a comment - Committed revision 899185.
        Hide
        Doug Cutting added a comment -

        This change causes a unit tests to fail:

        Testcase: testUnion took 0.062 sec
        Caused an ERROR
        Encoded data does not match. expected:<{"Foo"[:{}}]> but was:<{"Foo"[]>
        at org.apache.avro.TestSchema.checkJson(TestSchema.java:389)
        at org.apache.avro.TestSchema.testUnion(TestSchema.java:226)

        Please run 'ant clean test-java' before committing.

        Show
        Doug Cutting added a comment - This change causes a unit tests to fail: Testcase: testUnion took 0.062 sec Caused an ERROR Encoded data does not match. expected:<{"Foo" [:{}}] > but was:<{"Foo"[]> at org.apache.avro.TestSchema.checkJson(TestSchema.java:389) at org.apache.avro.TestSchema.testUnion(TestSchema.java:226) Please run 'ant clean test-java' before committing.
        Hide
        Thiruvalluvan M. G. added a comment -

        Committed revision 899520. Restored Parser.processImplicitActions() as it is still required.

        Show
        Thiruvalluvan M. G. added a comment - Committed revision 899520. Restored Parser.processImplicitActions() as it is still required.

          People

          • Assignee:
            Thiruvalluvan M. G.
            Reporter:
            Thiruvalluvan M. G.
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development