Avro
  1. Avro
  2. AVRO-503

ArrayIndexOutOfBoundsException with nested maps

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 1.3.2
    • Fix Version/s: None
    • Component/s: java, python
    • Labels:
      None

      Description

      I'm attempting to add an avro equivalent to Cassandra's batch_mutate() method (thrift def here: https://svn.apache.org/repos/asf/cassandra/trunk/interface/cassandra.thrift). This is the first time I've attempted to nest maps within maps, and I'm assuming it's related since that seems like the only difference to the working examples I have.

      Attached is a tarball that includes the protocol I'm using along with java and python examples (src/BatchMutate.java and py/batch_mutate.py) that reproduce the exceptions I'm seeing.

      Running the java example produces:

       
      Exception in thread "main" java.lang.ArrayIndexOutOfBoundsException: -1
      	at org.apache.avro.io.parsing.Symbol$Alternative.getSymbol(Symbol.java:364)
      	at org.apache.avro.io.ResolvingDecoder.doAction(ResolvingDecoder.java:191)
      	at org.apache.avro.io.parsing.Parser.advance(Parser.java:88)
      	at org.apache.avro.io.ResolvingDecoder.readIndex(ResolvingDecoder.java:168)
      	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:81)
      	at org.apache.avro.generic.GenericDatumReader.readRecord(GenericDatumReader.java:105)
      	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:77)
      	at org.apache.avro.generic.GenericDatumReader.read(GenericDatumReader.java:70)
      	at org.apache.avro.ipc.Requestor.readHandshake(Requestor.java:181)
      	at org.apache.avro.ipc.Requestor.request(Requestor.java:116)
      	at org.apache.avro.specific.SpecificRequestor.invoke(SpecificRequestor.java:52)
      	at $Proxy0.batch_mutate(Unknown Source)
      	at BatchMutate.main(BatchMutate.java:43)
      

      Running the python examples produces:

       
      Traceback (most recent call last):
        File "/usr/lib/pymodules/python2.5/nose/case.py", line 183, in runTest
          self.test(*self.arg)
        File "/home/eevans/dev/src/git/cassandra/test/system/test_avro_server.py", line 161, in test_batch_mutate
          self.client.request('batch_mutate', params)
        File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 146, in request
          return self.request(message_name, request_datum)
        File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 142, in request
          call_response_exists = self.read_handshake_response(buffer_decoder)
        File "/usr/local/lib/python2.5/site-packages/avro/ipc.py", line 188, in read_handshake_response
          handshake_response = HANDSHAKE_REQUESTOR_READER.read(decoder)
        File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 411, in read
          return self.read_data(self.writers_schema, self.readers_schema, decoder)
        File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 455, in read_data
          return self.read_record(writers_schema, readers_schema, decoder)
        File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 643, in read_record
          field_val = self.read_data(field.type, readers_field.type, decoder)
        File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 453, in read_data
          return self.read_union(writers_schema, readers_schema, decoder)
        File "/usr/local/lib/python2.5/site-packages/avro/io.py", line 608, in read_union
          selected_writers_schema = writers_schema.schemas[index_of_schema]
      IndexError: list index out of range
      
      1. sample_bugfix.patch
        3 kB
        Eric Evans
      2. cass.tar.gz
        1.96 MB
        Eric Evans

        Issue Links

          Activity

          Eric Evans created issue -
          Eric Evans made changes -
          Field Original Value New Value
          Attachment cass.tar.gz [ 12440724 ]
          Hide
          Jeff Hodges added a comment -

          With a small change to the python client (a la AVRO-496), this is reproducible with trunk.

          Show
          Jeff Hodges added a comment - With a small change to the python client (a la AVRO-496 ), this is reproducible with trunk.
          Hide
          Jeff Hodges added a comment -

          Oh, and to get this up and running I did this:

          cd $AVRO_PROJECT
          wget https://issues.apache.org/jira/secure/attachment/12440724/cass.tar.gz --no-check-certificate
          tar xzvf cass.tar.gz
          cd cass
          mkdir -p build/classes

          . clspth
          find . -name "*.java" | xargs javac -d build/classes
          ./start

          I'm sure there's a quicker way but that worked. Note the sourcing of clspth.

          Show
          Jeff Hodges added a comment - Oh, and to get this up and running I did this: cd $AVRO_PROJECT wget https://issues.apache.org/jira/secure/attachment/12440724/cass.tar.gz --no-check-certificate tar xzvf cass.tar.gz cd cass mkdir -p build/classes . clspth find . -name "*.java" | xargs javac -d build/classes ./start I'm sure there's a quicker way but that worked. Note the sourcing of clspth.
          Hide
          Jeff Hodges added a comment -

          Digging deeper and using the avro jar in trunk, I find this curious value for the IO object in the decoder passed to HANDSHAKE_REQUESTOR_READER.read() in the python side.

          ? org.apache.avro.AvroRuntimeException: Not in union ["string",{"type":"error","name":"InvalidRequestException","namespace":"org.apache.cassandra.avro","fields":[

          Unknown macro: {"name"}

          ]},{"type":"error","name":"UnavailableException","fields":[

          Unknown macro: {"name"}

          ]},{"type":"error","name":"TimedOutException","fields":[

          Unknown macro: {"name"}

          ]}]: org.apache.avro.AvroRuntimeException: java.lang.NoSuchMethodException: org.apache.cassandra.avro.CassandraServer.batch_mutate(org.apache.avro.util.Utf8, java.util.Map, org.apache.cassandra.avro.ConsistencyLevel)

          Strange, strange.

          Show
          Jeff Hodges added a comment - Digging deeper and using the avro jar in trunk, I find this curious value for the IO object in the decoder passed to HANDSHAKE_REQUESTOR_READER.read() in the python side. ? org.apache.avro.AvroRuntimeException: Not in union ["string",{"type":"error","name":"InvalidRequestException","namespace":"org.apache.cassandra.avro","fields":[ Unknown macro: {"name"} ]},{"type":"error","name":"UnavailableException","fields":[ Unknown macro: {"name"} ]},{"type":"error","name":"TimedOutException","fields":[ Unknown macro: {"name"} ]}]: org.apache.avro.AvroRuntimeException: java.lang.NoSuchMethodException: org.apache.cassandra.avro.CassandraServer.batch_mutate(org.apache.avro.util.Utf8, java.util.Map, org.apache.cassandra.avro.ConsistencyLevel) Strange, strange.
          Hide
          Jeff Hodges added a comment -

          Okay! So! Eric!

          It looks like your SpecificResponder instantiation was wrong! You're calling it as:

          SpecificResponder responder = new SpecificResponder(Cassandra.class, new CassandraServer());

          but CassandraServer doesn't implement the methods you need! You implemented them in CassandraImpl. The call should be:

          SpecificResponder responder = new SpecificResponder(Cassandra.class, new CassandraImpl());

          Now, that fixes this problem, but the real problem is that java side isn't handling AvroRuntimeExceptions correctly! In fact, the java side seems to just spitting out the error text for any error raised back on the wire instead of doing something "safe". The strange error you were seeing is because the error text (mentioned in the last comment) is (obviously) an invalid avro response!

          So, we need to do a little more work around how we handle errors in the RPC layer. And you have an easy bug to fix.

          Show
          Jeff Hodges added a comment - Okay! So! Eric! It looks like your SpecificResponder instantiation was wrong! You're calling it as: SpecificResponder responder = new SpecificResponder(Cassandra.class, new CassandraServer()); but CassandraServer doesn't implement the methods you need! You implemented them in CassandraImpl. The call should be: SpecificResponder responder = new SpecificResponder(Cassandra.class, new CassandraImpl()); Now, that fixes this problem, but the real problem is that java side isn't handling AvroRuntimeExceptions correctly! In fact, the java side seems to just spitting out the error text for any error raised back on the wire instead of doing something "safe". The strange error you were seeing is because the error text (mentioned in the last comment) is (obviously) an invalid avro response! So, we need to do a little more work around how we handle errors in the RPC layer. And you have an easy bug to fix.
          Jeff Hodges made changes -
          Link This issue incorporates AVRO-505 [ AVRO-505 ]
          Hide
          Jeff Hodges added a comment -

          Bonus for the Java hackers: Is there some generics magic (or even just runtime checks) we can do to prevent this accidental misuse of the SpecificResponder constructor?

          Show
          Jeff Hodges added a comment - Bonus for the Java hackers: Is there some generics magic (or even just runtime checks) we can do to prevent this accidental misuse of the SpecificResponder constructor?
          Jeff Hodges made changes -
          Link This issue incorporates AVRO-506 [ AVRO-506 ]
          Hide
          Eric Evans added a comment -

          but CassandraServer doesn't implement the methods you need! You implemented them in CassandraImpl.

          Gah. This was introduced in the process of creating the stand-alone example (that's where "CassandraServer" came from in fact). :/ I also just noticed that the Mutations array was empty (nothing is ever added to it).

          Show
          Eric Evans added a comment - but CassandraServer doesn't implement the methods you need! You implemented them in CassandraImpl. Gah. This was introduced in the process of creating the stand-alone example (that's where "CassandraServer" came from in fact). :/ I also just noticed that the Mutations array was empty (nothing is ever added to it).
          Hide
          Eric Evans added a comment -

          Ok, the attached patch (sample_bugfix.patch) fixes the HTTPTransceiver initialization for the Python example, and uses the correct instance when constructing a SpecificResponder in the Java example.

          With this applied the Java test-case (src/BatchMutate.java) now works against the Java test server (CassandraServer), but it still fails against the full implementation in Cassandra (same protocol schema). Here is the exception thrown on the RPC server (not sure why I didn't include that before):

          org.apache.avro.AvroRuntimeException: Not in union ["string",{"type":"error","name":"InvalidRequestException","namespace":"org.apache.cassandra.avro","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"UnavailableException","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"TimedOutException","fields":[{"name":"why","type":["string","null"]}]}]: java.lang.NullPointerException
          	at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:340)
          	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67)
          	at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55)
          	at org.apache.avro.specific.SpecificResponder.writeError(SpecificResponder.java:81)
          	at org.apache.avro.ipc.Responder.respond(Responder.java:137)
          	at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48)
          	at javax.servlet.http.HttpServlet.service(HttpServlet.java:727)
          	at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
          	at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511)
          	at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390)
          	at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765)
          	at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152)
          	at org.mortbay.jetty.Server.handle(Server.java:326)
          	at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:536)
          	at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:930)
          	at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747)
          	at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218)
          	at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405)
          	at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228)
          	at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582)
          

          Jeff, since you've been known to get your Cassandra on, I went ahead and committed the code I had to trunk/ there, in case that helps.

          Show
          Eric Evans added a comment - Ok, the attached patch (sample_bugfix.patch) fixes the HTTPTransceiver initialization for the Python example, and uses the correct instance when constructing a SpecificResponder in the Java example. With this applied the Java test-case (src/BatchMutate.java) now works against the Java test server (CassandraServer), but it still fails against the full implementation in Cassandra (same protocol schema). Here is the exception thrown on the RPC server (not sure why I didn't include that before): org.apache.avro.AvroRuntimeException: Not in union ["string",{"type":"error","name":"InvalidRequestException","namespace":"org.apache.cassandra.avro","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"UnavailableException","fields":[{"name":"why","type":["string","null"]}]},{"type":"error","name":"TimedOutException","fields":[{"name":"why","type":["string","null"]}]}]: java.lang.NullPointerException at org.apache.avro.generic.GenericData.resolveUnion(GenericData.java:340) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:67) at org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:55) at org.apache.avro.specific.SpecificResponder.writeError(SpecificResponder.java:81) at org.apache.avro.ipc.Responder.respond(Responder.java:137) at org.apache.avro.ipc.ResponderServlet.doPost(ResponderServlet.java:48) at javax.servlet.http.HttpServlet.service(HttpServlet.java:727) at javax.servlet.http.HttpServlet.service(HttpServlet.java:820) at org.mortbay.jetty.servlet.ServletHolder.handle(ServletHolder.java:511) at org.mortbay.jetty.servlet.ServletHandler.handle(ServletHandler.java:390) at org.mortbay.jetty.handler.ContextHandler.handle(ContextHandler.java:765) at org.mortbay.jetty.handler.HandlerWrapper.handle(HandlerWrapper.java:152) at org.mortbay.jetty.Server.handle(Server.java:326) at org.mortbay.jetty.HttpConnection.handleRequest(HttpConnection.java:536) at org.mortbay.jetty.HttpConnection$RequestHandler.content(HttpConnection.java:930) at org.mortbay.jetty.HttpParser.parseNext(HttpParser.java:747) at org.mortbay.jetty.HttpParser.parseAvailable(HttpParser.java:218) at org.mortbay.jetty.HttpConnection.handle(HttpConnection.java:405) at org.mortbay.jetty.bio.SocketConnector$Connection.run(SocketConnector.java:228) at org.mortbay.thread.QueuedThreadPool$PoolThread.run(QueuedThreadPool.java:582) Jeff, since you've been known to get your Cassandra on, I went ahead and committed the code I had to trunk/ there, in case that helps.
          Eric Evans made changes -
          Attachment sample_bugfix.patch [ 12440779 ]
          Hide
          Jeff Hodges added a comment -

          Totally does. Thanks, Eric. I'll take a look.

          Show
          Jeff Hodges added a comment - Totally does. Thanks, Eric. I'll take a look.

            People

            • Assignee:
              Unassigned
              Reporter:
              Eric Evans
            • Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

              • Created:
                Updated:

                Development