Avro
  1. Avro
  2. AVRO-864

NoSuchFieldException when emitting record with array of union of records in mapred

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.5.2
    • Component/s: java
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      I tried to convert data records currently in a custom format into avro records via a map-only job. To allow for optional record fields I followed the workaround sketched by Doug in AVRO-519 using the generic API.

      When emitting the resulting records a "NoSuchFieldException" is thrown when trying to resolve the union types: Only the methods declared in ReflectData seem to be called, the implementation in GenericData is skipped.

      1. AVRO-864.diff
        7 kB
        Doug Cutting
      2. AVRO-864.diff
        7 kB
        Isabel Drost-Fromm

        Activity

        Hide
        Isabel Drost-Fromm added a comment -

        Patch illustrating the issue - contains a unit test to reproduce the problem as well as two small changes that so far have fixed the problem for me.

        Big Thanks+kiss to Thilo Fromm for helping me debug this.

        Show
        Isabel Drost-Fromm added a comment - Patch illustrating the issue - contains a unit test to reproduce the problem as well as two small changes that so far have fixed the problem for me. Big Thanks+kiss to Thilo Fromm for helping me debug this.
        Hide
        Doug Cutting added a comment -

        Here's a slightly different version of the patch. Instead of catching exceptions it calls super directly for generic and specific records. The provided unit test still passes.

        Show
        Doug Cutting added a comment - Here's a slightly different version of the patch. Instead of catching exceptions it calls super directly for generic and specific records. The provided unit test still passes.
        Hide
        Doug Cutting added a comment -

        I will commit this soon unless there are objections.

        Show
        Doug Cutting added a comment - I will commit this soon unless there are objections.
        Hide
        Doug Cutting added a comment -

        I committed this. Thanks, Isabel!

        Show
        Doug Cutting added a comment - I committed this. Thanks, Isabel!

          People

          • Assignee:
            Isabel Drost-Fromm
            Reporter:
            Isabel Drost-Fromm
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development