Uploaded image for project: 'Cassandra'
  1. Cassandra
  2. CASSANDRA-2255

ColumnFamilyOutputFormat drops mutations when batches fill up.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Fix Version/s: 0.7.3
    • Component/s: None
    • Labels:
      None

      Description

      queue.poll() takes a mutation,
      but then the batch is already full,
      so the while loop exits, ant the mutation we just got is dropped.

      1. 2255-patch-2.txt
        1 kB
        Jeremy Hanna
      2. 0001_Stop_dropping_mutations.txt
        1 kB
        Eldon Stegall

        Activity

        Hide
        eldondev Eldon Stegall added a comment -

        This should help.

        Show
        eldondev Eldon Stegall added a comment - This should help.
        Hide
        eldondev Eldon Stegall added a comment -

        Should apply cleanly to 0.7.0 tag

        Show
        eldondev Eldon Stegall added a comment - Should apply cleanly to 0.7.0 tag
        Hide
        brandon.williams Brandon Williams added a comment -

        Committed, thanks!

        Show
        brandon.williams Brandon Williams added a comment - Committed, thanks!
        Hide
        hudson Hudson added a comment -

        Integrated in Cassandra-0.7 #332 (See https://hudson.apache.org/hudson/job/Cassandra-0.7/332/)
        CFRW no longer loses mutations.
        Patch by Eldon Stegall, reviewed by Stu Hood and brandonwilliams for
        CASSANDRA-2255

        Show
        hudson Hudson added a comment - Integrated in Cassandra-0.7 #332 (See https://hudson.apache.org/hudson/job/Cassandra-0.7/332/ ) CFRW no longer loses mutations. Patch by Eldon Stegall, reviewed by Stu Hood and brandonwilliams for CASSANDRA-2255
        Hide
        jbellis Jonathan Ellis added a comment -

        Doesn't this mean batch sizes can be arbitrarily large? Not only will this cause latency spikes on the server but you could OOM building the monster batch.

        Show
        jbellis Jonathan Ellis added a comment - Doesn't this mean batch sizes can be arbitrarily large? Not only will this cause latency spikes on the server but you could OOM building the monster batch.
        Hide
        jbellis Jonathan Ellis added a comment -

        also: the mutation = null; line is dead code.

        Show
        jbellis Jonathan Ellis added a comment - also: the mutation = null; line is dead code.
        Hide
        jbellis Jonathan Ellis added a comment -

        reverted

        Show
        jbellis Jonathan Ellis added a comment - reverted
        Hide
        jeromatron Jeremy Hanna added a comment -

        another go at the patch

        Show
        jeromatron Jeremy Hanna added a comment - another go at the patch
        Hide
        jbellis Jonathan Ellis added a comment -

        lgtm, committed (we'll try to get it in to 0.7.3 re-spin)

        Show
        jbellis Jonathan Ellis added a comment - lgtm, committed (we'll try to get it in to 0.7.3 re-spin)
        Hide
        hudson Hudson added a comment -

        Integrated in Cassandra-0.7 #334 (See https://hudson.apache.org/hudson/job/Cassandra-0.7/334/)
        fix Hadoop ColumnFamilyOutputFormat droppingof mutations
        patch by Eldon Stegall and Jeremy Hanna; reviewed by jbellis for CASSANDRA-2255

        Show
        hudson Hudson added a comment - Integrated in Cassandra-0.7 #334 (See https://hudson.apache.org/hudson/job/Cassandra-0.7/334/ ) fix Hadoop ColumnFamilyOutputFormat droppingof mutations patch by Eldon Stegall and Jeremy Hanna; reviewed by jbellis for CASSANDRA-2255

          People

          • Assignee:
            jeromatron Jeremy Hanna
            Reporter:
            eldondev Eldon Stegall
            Reviewer:
            Jonathan Ellis
          • Votes:
            1 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development