Cassandra
  1. Cassandra
  2. CASSANDRA-926

remove alternative RPC interface using Avro

    Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 0.8 beta 1
    • Component/s: Core
    • Labels:

      Description

      UPDATE: This ticket is now for the removal of the Avro interface, (see http://thread.gmane.org/gmane.comp.db.cassandra.client.devel/36 for background).

      Avro is data serialization and RPC framework similar to Thrift. It provides:

      • Rich data structures.
      • A compact, fast, binary data format.
      • A container file, to store persistent data.
      • Remote procedure call (RPC).
      • Simple integration with dynamic languages. Code generation is not required to read or write data files nor to use or implement RPC protocols. Code generation as an optional optimization, only worth implementing for statically typed languages.

      Cassandra's Avro interface is being structured in a way that closely mirrors the existing Thrift interface, both in terms of public facing API, and how it is implemented. GSOC students interested in this task should begin by familiarizing themselves with Cassandra's Thrift service (org.apache.cassandra.thrift).

      Note: This is a very large and long-running task so treat this as a meta-issue and add sub-tasks and/or blocking issues as appropriate.

        Issue Links

          Activity

          Hide
          Eric Evans added a comment -

          It's been suggested that we eliminate the avro rpc implementation entirely, see http://thread.gmane.org/gmane.comp.db.cassandra.client.devel/36 for the discussion.

          Show
          Eric Evans added a comment - It's been suggested that we eliminate the avro rpc implementation entirely, see http://thread.gmane.org/gmane.comp.db.cassandra.client.devel/36 for the discussion.
          Hide
          Eric Evans added a comment -

          Consensus seems to be that there is no longer sufficient interest in an Avro RPC interface, so let's remove it.

          Show
          Eric Evans added a comment - Consensus seems to be that there is no longer sufficient interest in an Avro RPC interface, so let's remove it.
          Hide
          Eric Evans added a comment -
          • 0001: adds record defs to internode.genavro that had been previously "included" from cassandra.genavro. Since the records where added to the org.apache.cassandra.db.migration.avro package, this patch also contains a lot of search-and-replace to update it from org.apache.cassandra.avro.
          • 0002: strips cassandra.genavro over everything not needed by the hadoop package.
          • 0003: removes the option to start an avro daemon from bin/cassandra
          • 0004: removes the functional tests.
          • 0005: removes (non-generated )source for the rpc interface.
          • 0006: moves the remaining records to the hadoop package.
          Show
          Eric Evans added a comment - 0001: adds record defs to internode.genavro that had been previously "included" from cassandra.genavro. Since the records where added to the org.apache.cassandra.db.migration.avro package, this patch also contains a lot of search-and-replace to update it from org.apache.cassandra.avro . 0002: strips cassandra.genavro over everything not needed by the hadoop package. 0003: removes the option to start an avro daemon from bin/cassandra 0004: removes the functional tests. 0005: removes (non-generated )source for the rpc interface. 0006: moves the remaining records to the hadoop package.
          Hide
          Jeremy Hanna added a comment - - edited

          Looks good. Two little things.

          • In contrib/word_count/src/WordCount.java, the package for the avro stuff needs to change from org.apache.cassandra.avro.XXX to org.apache.cassandra.hadoop.avro.XXX.
          • In contrib/hadoop_output_streaming/bin/streaming, line 35 needs to change to: "-file $cwd/../../../src/gen-java/org/apache/cassandra/hadoop/hadoop.avpr \"

          Other than that, looks good to me.

          Show
          Jeremy Hanna added a comment - - edited Looks good. Two little things. In contrib/word_count/src/WordCount.java, the package for the avro stuff needs to change from org.apache.cassandra.avro.XXX to org.apache.cassandra.hadoop.avro.XXX. In contrib/hadoop_output_streaming/bin/streaming, line 35 needs to change to: "-file $cwd/../../../src/gen-java/org/apache/cassandra/hadoop/hadoop.avpr \" Other than that, looks good to me.
          Hide
          Jeremy Hanna added a comment - - edited

          Based on the need for those contrib changes, we probably want to mention the need to update map/reduce and hadoop streaming code accordingly in NEWS.txt.

          Show
          Jeremy Hanna added a comment - - edited Based on the need for those contrib changes, we probably want to mention the need to update map/reduce and hadoop streaming code accordingly in NEWS.txt.
          Hide
          Eric Evans added a comment -

          Checked-in, including changes for the feedback above. Thanks Jeremy!

          Show
          Eric Evans added a comment - Checked-in, including changes for the feedback above. Thanks Jeremy!
          Hide
          Hudson added a comment -

          Integrated in Cassandra #673 (See https://hudson.apache.org/hudson/job/Cassandra/673/)
          update NEWS.txt to document avro package changes

          Patch by eevans for CASSANDRA-926
          missed references caught in review

          Patch by eevans for CASSANDRA-926
          move remaining avro records to hadoop package

          Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926
          remove avro rpc source

          Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926
          remove avro functional tests

          Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926
          remove option to launch avro rpc daemon

          Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926
          don't build schema for rpc interface

          Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926
          CASSANDRA-926 denormalize avro internode/rpc schema

          Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926

          Show
          Hudson added a comment - Integrated in Cassandra #673 (See https://hudson.apache.org/hudson/job/Cassandra/673/ ) update NEWS.txt to document avro package changes Patch by eevans for CASSANDRA-926 missed references caught in review Patch by eevans for CASSANDRA-926 move remaining avro records to hadoop package Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926 remove avro rpc source Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926 remove avro functional tests Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926 remove option to launch avro rpc daemon Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926 don't build schema for rpc interface Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926 CASSANDRA-926 denormalize avro internode/rpc schema Patch by eevans; reviewed by Jeremy Hanna for CASSANDRA-926

            People

            • Assignee:
              Eric Evans
              Reporter:
              Eric Evans
            • Votes:
              3 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development