Details

    • Type: Improvement Improvement
    • Status: Resolved
    • Priority: Minor Minor
    • Resolution: Fixed
    • Fix Version/s: 2.2.0 beta 1
    • Component/s: Configuration
    • Labels:
      None

      Description

      Related to https://issues.apache.org/jira/browse/CASSANDRA-6487

      Just education alone is not going to stop some of the largest batch sizes from being used. Just like we have a tombstone fail threshold, I propose that we have a batch size fail threshold.

      Maybe 10x warn?

      batch_fail_threshold: 50k

      1. 8011-trunk.txt
        6 kB
        Carl Yeksigian
      2. 8011-trunk-v2.txt
        10 kB
        Carl Yeksigian
      3. 8011-trunk-v3.txt
        10 kB
        Carl Yeksigian

        Issue Links

          Activity

          Hide
          Carl Yeksigian added a comment - - edited

          Added the option which fails on very large batches.

          batch_size_fail_threshold_in_kb: 50
          
          Show
          Carl Yeksigian added a comment - - edited Added the option which fails on very large batches. batch_size_fail_threshold_in_kb: 50
          Hide
          sankalp kohli added a comment -

          Some nits
          1) In your comment I think you mean "batch_size_fail_threshold_in_kb" and not "batch_size_file_threshold_in_kb"
          2) We should also expose changing this via JMX like we do for TombstoneFailureThreshold.
          3) When we encounter TombstoneFailureThreshold, we clearly write in the exception like " (see tombstone_warn_threshold)". We should add the same for this.

          Show
          sankalp kohli added a comment - Some nits 1) In your comment I think you mean "batch_size_fail_threshold_in_kb" and not "batch_size_file_threshold_in_kb" 2) We should also expose changing this via JMX like we do for TombstoneFailureThreshold. 3) When we encounter TombstoneFailureThreshold, we clearly write in the exception like " (see tombstone_warn_threshold)". We should add the same for this.
          Hide
          Carl Yeksigian added a comment -

          Adding a v2 which address sankalp kohli's comments.

          Show
          Carl Yeksigian added a comment - Adding a v2 which address sankalp kohli 's comments.
          Hide
          sankalp kohli added a comment -

          You are assigning the variable in DatabaseDescriptor. Why not do as it is done for similar stuff like tombstone_failure_threshold.
          In Config, you can make public volatile int batch_size_fail_threshold_in_kb = 50 like it is done for tombstone_failure_threshold.

          Show
          sankalp kohli added a comment - You are assigning the variable in DatabaseDescriptor. Why not do as it is done for similar stuff like tombstone_failure_threshold. In Config, you can make public volatile int batch_size_fail_threshold_in_kb = 50 like it is done for tombstone_failure_threshold.
          Hide
          Carl Yeksigian added a comment -

          Removed the variable in DatabaseDescriptor; just using Config.

          Show
          Carl Yeksigian added a comment - Removed the variable in DatabaseDescriptor; just using Config.
          Hide
          sankalp kohli added a comment -

          +1.

          Show
          sankalp kohli added a comment - +1.
          Hide
          Jonathan Ellis added a comment -

          committed

          Show
          Jonathan Ellis added a comment - committed
          Hide
          Benedict added a comment -

          This restriction seems to apply even when the batch covers only one partition. We shouldn't really be upset at users sending large batch updates that are for a single partition, as artificially separating the updates doesn't provide any benefit to Cassandra, and most likely pushes problems onto the user.

          Show
          Benedict added a comment - This restriction seems to apply even when the batch covers only one partition. We shouldn't really be upset at users sending large batch updates that are for a single partition, as artificially separating the updates doesn't provide any benefit to Cassandra, and most likely pushes problems onto the user.

            People

            • Assignee:
              Carl Yeksigian
              Reporter:
              Patrick McFadin
              Reviewer:
              sankalp kohli
            • Votes:
              1 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development