Details

    • New Feature
    • Status: Resolved
    • Low
    • Resolution: Fixed
    • 0.6
    • None
    • None

    Description

      This will allow using Thrift from hadoop jobs, which makes me a little sad but MessagingService is just not a good fit for how Hadoop wants to run jobs.

      Attachments

        1. 775-with-thrift.txt
          85 kB
          Jonathan Ellis
        2. 775.txt
          8 kB
          Jonathan Ellis

        Issue Links

          Activity

            stuhood Stu Hood added a comment -

            It really is too bad Thrift doesn't support method overloading... sigh.

            After calling get_range_slices with a KeyRange limited to N rows, how do you make another call to get rows starting from N?

            stuhood Stu Hood added a comment - It really is too bad Thrift doesn't support method overloading... sigh. After calling get_range_slices with a KeyRange limited to N rows, how do you make another call to get rows starting from N?
            jbellis Jonathan Ellis added a comment -

            I don't understand the question... using start_key and end_key works like "classic" RQ.

            jbellis Jonathan Ellis added a comment - I don't understand the question... using start_key and end_key works like "classic" RQ.
            stuhood Stu Hood added a comment - - edited

            > I don't understand the question... using start_key and end_key works like "classic" RQ.
            Right, but since you don't get Token strings back when you query, you can't requery using a new Token.

            EDIT: Hmm, I guess for the second query you can use a start_key and a finish_token. Is that the idea?
            EDIT2: Actually, that would be invalid with this patch, so the question stands.

            stuhood Stu Hood added a comment - - edited > I don't understand the question... using start_key and end_key works like "classic" RQ. Right, but since you don't get Token strings back when you query, you can't requery using a new Token. EDIT: Hmm, I guess for the second query you can use a start_key and a finish_token. Is that the idea? EDIT2: Actually, that would be invalid with this patch, so the question stands.
            jbellis Jonathan Ellis added a comment -

            it's really only intended for hadoop, which parallelizes the queries anyway

            jbellis Jonathan Ellis added a comment - it's really only intended for hadoop, which parallelizes the queries anyway
            jbellis Jonathan Ellis added a comment -

            (but, anyone who knows how partitioner works, which is trivial w/ java clients and not hard for others, would be welcome to compute tokens from keys)

            jbellis Jonathan Ellis added a comment - (but, anyone who knows how partitioner works, which is trivial w/ java clients and not hard for others, would be welcome to compute tokens from keys)
            stuhood Stu Hood added a comment -

            Hmm, I'd like to find a way that puts clients on more equal footing if possible. Requiring the key->token conversion on the client side for paging is ugly.

            stuhood Stu Hood added a comment - Hmm, I'd like to find a way that puts clients on more equal footing if possible. Requiring the key->token conversion on the client side for paging is ugly.
            stuhood Stu Hood added a comment -

            What about adding a Token to KeySlice?

            stuhood Stu Hood added a comment - What about adding a Token to KeySlice?
            jbellis Jonathan Ellis added a comment -

            if you're not doing parallel queries you have no need to use the token api over keys anyway. so i'd be against adding cruft to the api in the absence of an actual use case for it.

            jbellis Jonathan Ellis added a comment - if you're not doing parallel queries you have no need to use the token api over keys anyway. so i'd be against adding cruft to the api in the absence of an actual use case for it.
            jbellis Jonathan Ellis added a comment -

            rebased.

            i also note that if adding a token to keyslice does end up being the best way to solve really large numbers of keys, it can be done by adding an optional field to the struct w/o breaking backwards compatibility.

            jbellis Jonathan Ellis added a comment - rebased. i also note that if adding a token to keyslice does end up being the best way to solve really large numbers of keys, it can be done by adding an optional field to the struct w/o breaking backwards compatibility.
            stuhood Stu Hood added a comment - - edited

            Would you mind adding a compiled version of KeyRange to the patch? Trying to find a thrift version to rebuild the server interface with is painful.

            EDIT: And the changes to Cassandra.Client.

            stuhood Stu Hood added a comment - - edited Would you mind adding a compiled version of KeyRange to the patch? Trying to find a thrift version to rebuild the server interface with is painful. EDIT: And the changes to Cassandra.Client.
            jbellis Jonathan Ellis added a comment -

            we always include the thrift revision in the lib/ file so it's easy to svn up to the right one. but here is a patch w/ the thrift changes bundled.

            jbellis Jonathan Ellis added a comment - we always include the thrift revision in the lib/ file so it's easy to svn up to the right one. but here is a patch w/ the thrift changes bundled.
            jbellis Jonathan Ellis added a comment -

            From IRC:

            stuhood: my +1 is to get both merged, and then open a new ticket to fix getSplits and add paging

            jbellis Jonathan Ellis added a comment - From IRC: stuhood: my +1 is to get both merged, and then open a new ticket to fix getSplits and add paging
            jbellis Jonathan Ellis added a comment -

            committed. Created CASSANDRA-789 for paging.

            jbellis Jonathan Ellis added a comment - committed. Created CASSANDRA-789 for paging.
            hudson Hudson added a comment -
            hudson Hudson added a comment - Integrated in Cassandra #357 (See http://hudson.zones.apache.org/hudson/job/Cassandra/357/ )

            People

              jbellis Jonathan Ellis
              jbellis Jonathan Ellis
              Jonathan Ellis
              Votes:
              0 Vote for this issue
              Watchers:
              1 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: