Hadoop Common
  1. Hadoop Common
  2. HADOOP-6782

TestAvroRpc fails with avro-1.3.1 and avro-1.3.2

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.21.0
    • Component/s: None
    • Labels:
      None

      Description

      I tried to upgrade Hadoop-common to use avro-1.3.1. TestAvroRpc fails with following exceptions.
      Same exception is seen with avro-1.3.2 as well.

      org.apache.avro.AvroRuntimeException: Not in union ["string"]: org.apache.avro.AvroRuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.ipc.TestAvroRpc$TestImpl.echo(java.lang.String)
      at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:340)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:116)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
      at org.apache.avro.specific.SpecificResponder.writeError(SpecificResponder.java:81)
      at org.apache.avro.ipc.Responder.respond(Responder.java:137)
      at org.apache.hadoop.ipc.AvroRpcEngine$TunnelResponder.call(AvroRpcEngine.java:183)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:342)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1350)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1346)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:738)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1344)
      2010-05-20 13:50:39,227 INFO ipc.Server (Server.java:run(1358)) - IPC Server handler 0 on 54672, call call(org.apache.hadoop.ipc.AvroRpcEngine$BufferListWritable@7bcd280b) from 127.0.0.1:54673: error: java.io.IOException: org.apache.avro.AvroRuntimeException: Not in union ["string"]: org.apache.avro.AvroRuntimeException: Not in union ["string"]: org.apache.avro.AvroRuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.ipc.TestAvroRpc$TestImpl.echo(java.lang.String)
      java.io.IOException: org.apache.avro.AvroRuntimeException: Not in union ["string"]: org.apache.avro.AvroRuntimeException: Not in union ["string"]: org.apache.avro.AvroRuntimeException: java.lang.NoSuchMethodException: org.apache.hadoop.ipc.TestAvroRpc$TestImpl.echo(java.lang.String)
      at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:340)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
      at org.apache.avro.reflect.ReflectDatumWriter.write(ReflectDatumWriter.java:116)
      at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
      at org.apache.avro.specific.SpecificResponder.writeError(SpecificResponder.java:81)
      at org.apache.avro.ipc.Responder.respond(Responder.java:146)
      at org.apache.hadoop.ipc.AvroRpcEngine$TunnelResponder.call(AvroRpcEngine.java:183)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.ipc.WritableRpcEngine$Server.call(WritableRpcEngine.java:342)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1350)
      at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1346)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:738)
      at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1344)

        Issue Links

          Activity

          Hide
          Scott Carey added a comment -

          I think I have seen something similar with Avro in the past, when using the Specific API.

          Does this stop if the Datum that is being written is not recycled? I think it is related to recycling an object with a Union and having the underlying type of the union differ from one use to another. Is the union above a union of "string" and something other than "null"?

          Show
          Scott Carey added a comment - I think I have seen something similar with Avro in the past, when using the Specific API. Does this stop if the Datum that is being written is not recycled? I think it is related to recycling an object with a Union and having the underlying type of the union differ from one use to another. Is the union above a union of "string" and something other than "null"?
          Hide
          Doug Cutting added a comment -

          Here's a patch that fixes this.

          Show
          Doug Cutting added a comment - Here's a patch that fixes this.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12445109/HADOOP-6782.patch
          against trunk revision 946487.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 6 new or modified tests.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/testReport/
          Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/artifact/trunk/build/test/checkstyle-errors.html
          Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12445109/HADOOP-6782.patch against trunk revision 946487. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 6 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch-h4.grid.sp2.yahoo.net/535/console This message is automatically generated.
          Hide
          Jitendra Nath Pandey added a comment -

          Committed by Jakob as part of HADOOP-6581.

          Show
          Jitendra Nath Pandey added a comment - Committed by Jakob as part of HADOOP-6581 .
          Hide
          Tom White added a comment -

          I was hitting this in 0.21 so I've committed the patch there.

          Show
          Tom White added a comment - I was hitting this in 0.21 so I've committed the patch there.

            People

            • Assignee:
              Doug Cutting
              Reporter:
              Jitendra Nath Pandey
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development