Uploaded image for project: 'Accumulo'
  1. Accumulo
  2. ACCUMULO-466

add a default constraint to limit the size of keys

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 1.6.0
    • Component/s: master
    • Labels:
      None

      Description

      If a user mistakenly inserts extremely large keys, the tablet server will eventually run out of memory just loading indexes. This is almost certainly a mistake. Add a constraint, by default, to new tables, which will limit the size of a key to a megabyte, for example, to catch these kinds of errors earlier. If the user intends to use very large keys, they can always remove the constraint.

      1. ACCUMULO-466.1.patch
        12 kB
        Corey J. Nolet

        Activity

        Hide
        hudson Hudson added a comment -

        Integrated in Accumulo-Trunk #851 (See https://builds.apache.org/job/Accumulo-Trunk/851/)
        ACCUMULO-466 commit Corey Nolet's patch to add a default constraint limit the size of keys (Revision 1478404)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraint.java
        • /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/IteratorUtil.java
        • /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints
        • /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraintTest.java
        • /accumulo/trunk/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java
        • /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/ShellServerTest.java
        • /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java
        Show
        hudson Hudson added a comment - Integrated in Accumulo-Trunk #851 (See https://builds.apache.org/job/Accumulo-Trunk/851/ ) ACCUMULO-466 commit Corey Nolet's patch to add a default constraint limit the size of keys (Revision 1478404) Result = SUCCESS ecn : Files : /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraint.java /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/IteratorUtil.java /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraintTest.java /accumulo/trunk/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/ShellServerTest.java /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java
        Hide
        hudson Hudson added a comment -

        Integrated in Accumulo-Trunk-Hadoop-2.0 #209 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/209/)
        ACCUMULO-466 commit Corey Nolet's patch to add a default constraint limit the size of keys (Revision 1478404)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraint.java
        • /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/IteratorUtil.java
        • /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints
        • /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraintTest.java
        • /accumulo/trunk/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java
        • /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/ShellServerTest.java
        • /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java
        Show
        hudson Hudson added a comment - Integrated in Accumulo-Trunk-Hadoop-2.0 #209 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/209/ ) ACCUMULO-466 commit Corey Nolet's patch to add a default constraint limit the size of keys (Revision 1478404) Result = SUCCESS ecn : Files : /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraint.java /accumulo/trunk/core/src/main/java/org/apache/accumulo/core/iterators/IteratorUtil.java /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints /accumulo/trunk/core/src/test/java/org/apache/accumulo/core/constraints/DefaultKeySizeConstraintTest.java /accumulo/trunk/proxy/src/test/java/org/apache/accumulo/proxy/SimpleTest.java /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/ShellServerTest.java /accumulo/trunk/test/src/test/java/org/apache/accumulo/test/TableOperationsIT.java
        Hide
        ctubbsii Christopher Tubbs added a comment -

        I think it's too late for 1.5. It'd be a nice 1.6 feature.

        Show
        ctubbsii Christopher Tubbs added a comment - I think it's too late for 1.5. It'd be a nice 1.6 feature.
        Hide
        kturner Keith Turner added a comment -

        Could also make the Accumulo file output format do something similar. It would throw an error if you try to write a really large key. If the user really wants to do this, they can configure the output format to accept larger keys.

        Should this be a 1.5 fix?

        Show
        kturner Keith Turner added a comment - Could also make the Accumulo file output format do something similar. It would throw an error if you try to write a really large key. If the user really wants to do this, they can configure the output format to accept larger keys. Should this be a 1.5 fix?

          People

          • Assignee:
            sonixbp Corey J. Nolet
            Reporter:
            ecn Eric Newton
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development