Accumulo
  1. Accumulo
  2. ACCUMULO-1044

bulk imported files showing up in metadata after bulk import fails

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 1.4.2
    • Fix Version/s: 1.5.0
    • Component/s: master, tserver
    • Labels:

      Description

      Bulk import fails. The file is moved to the failures directory.

      But references in the !METADATA table remain.

        Activity

        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.4.x #276 (See https://builds.apache.org/job/Accumulo-1.4.x/276/)
        ACCUMULO-1044 properly extract a column qualifier as a string (Revision 1450271)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.4.x #276 (See https://builds.apache.org/job/Accumulo-1.4.x/276/ ) ACCUMULO-1044 properly extract a column qualifier as a string (Revision 1450271) Result = SUCCESS ecn : Files : /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk-Hadoop-2.0 #103 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/103/)
        ACCUMULO-1044 properly extract a column qualifier as a string (Revision 1450274)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk-Hadoop-2.0 #103 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/103/ ) ACCUMULO-1044 properly extract a column qualifier as a string (Revision 1450274) Result = SUCCESS ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java /accumulo/trunk/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk #745 (See https://builds.apache.org/job/Accumulo-Trunk/745/)
        ACCUMULO-1044 properly extract a column qualifier as a string (Revision 1450274)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk #745 (See https://builds.apache.org/job/Accumulo-Trunk/745/ ) ACCUMULO-1044 properly extract a column qualifier as a string (Revision 1450274) Result = SUCCESS ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java /accumulo/trunk/src
        Hide
        Keith Turner added a comment -

        the change may have no effect, ArrayByteSequence has a toString method

        Show
        Keith Turner added a comment - the change may have no effect, ArrayByteSequence has a toString method
        Hide
        Keith Turner added a comment -

        may want to add toString to ByteSequence to make things more clear.... ArrayByteSequence is probably the only implemetation used by Accumulo code

        Show
        Keith Turner added a comment - may want to add toString to ByteSequence to make things more clear.... ArrayByteSequence is probably the only implemetation used by Accumulo code
        Hide
        Eric Newton added a comment -

        If a tablet fails to update the !METADATA table with the bulk entry due to RPC timeouts, the update might execute outside of the lock/transaction counts. Adding an arbitrator check in the MetadataConstraints might help.

        Show
        Eric Newton added a comment - If a tablet fails to update the !METADATA table with the bulk entry due to RPC timeouts, the update might execute outside of the lock/transaction counts. Adding an arbitrator check in the MetadataConstraints might help.
        Hide
        Eric Newton added a comment -

        This may be impossible to fix without a complete restructuring of bulk import.

        There's a race condition between the update of the metadata with a bulk-file-loaded flag, and the closing of the transaction. The current code keeps this window very small, but it is still possible.

        Another "fix" is to never move files to the failed directory: always copy them. However, the race condition is just moved from the Master to the Garbage Collector.

        The work-around now is to increase the number of retries to a very high number.

        Show
        Eric Newton added a comment - This may be impossible to fix without a complete restructuring of bulk import. There's a race condition between the update of the metadata with a bulk-file-loaded flag, and the closing of the transaction. The current code keeps this window very small, but it is still possible. Another "fix" is to never move files to the failed directory: always copy them. However, the race condition is just moved from the Master to the Garbage Collector. The work-around now is to increase the number of retries to a very high number.
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5 #61 (See https://builds.apache.org/job/Accumulo-1.5/61/)
        ACCUMULO-1044 add a metadata table constraint to require the bulk load transaction to be alive when writing the bulk-loaded flags (Revision 1463552)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5 #61 (See https://builds.apache.org/job/Accumulo-1.5/61/ ) ACCUMULO-1044 add a metadata table constraint to require the bulk load transaction to be alive when writing the bulk-loaded flags (Revision 1463552) Result = SUCCESS ecn : Files : /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5-Hadoop-2.0 #59 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/59/)
        ACCUMULO-1044 add a metadata table constraint to require the bulk load transaction to be alive when writing the bulk-loaded flags (Revision 1463552)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5-Hadoop-2.0 #59 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/59/ ) ACCUMULO-1044 add a metadata table constraint to require the bulk load transaction to be alive when writing the bulk-loaded flags (Revision 1463552) Result = SUCCESS ecn : Files : /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        Hide
        Keith Turner added a comment -

        commit 1463552 checks to see if METADATA_TIME_COLUMN is set to determine if something is a split mutation. Bulk load mutations also set METADATA_TIME_COLUMN

         if (new ColumnFQ(update).equals(Constants.METADATA_TIME_COLUMN)){
           isSplitMutation = true;
        
        Show
        Keith Turner added a comment - commit 1463552 checks to see if METADATA_TIME_COLUMN is set to determine if something is a split mutation. Bulk load mutations also set METADATA_TIME_COLUMN if ( new ColumnFQ(update).equals(Constants.METADATA_TIME_COLUMN)){ isSplitMutation = true ;
        Hide
        Eric Newton added a comment -

        MetadataBulkLoadFilter is running too quickly.

        Bulk loading is a pretty intricate dance of checks:

        • The master distributes the bulk files to tablet servers
        • Tablet servers attempt to determine where the tablets go based on their index information
        • When a tablet incorporates the new file, it has to update the !METADATA table
        • The tablet marks the fact that it has loaded the file
        • When the master completes the bulk import, it removes a transaction id in zookeeper
        • Tablets will not load files once the transaction id is removed
        • The master than asks if anyone is still working on that transaction id
        • Once the master has verified that nobody is doing anything on behalf of the transaction, it removes the flags that indicate that the file loaded
        • Because splits can occur while the master is removing markers, there's a METADATA filter to remove them

        Here's the problem:

        • the master removes the transaction id
        • the metadata table major compacts, sees the id is missing, so it removes the flags: this is bad
        • the master continues to wait for threads to stop doing work for the transaction
        • the master then sees that there are no references to the tablet and moves it to the failed directory
        Show
        Eric Newton added a comment - MetadataBulkLoadFilter is running too quickly. Bulk loading is a pretty intricate dance of checks: The master distributes the bulk files to tablet servers Tablet servers attempt to determine where the tablets go based on their index information When a tablet incorporates the new file, it has to update the !METADATA table The tablet marks the fact that it has loaded the file When the master completes the bulk import, it removes a transaction id in zookeeper Tablets will not load files once the transaction id is removed The master than asks if anyone is still working on that transaction id Once the master has verified that nobody is doing anything on behalf of the transaction, it removes the flags that indicate that the file loaded Because splits can occur while the master is removing markers, there's a METADATA filter to remove them Here's the problem: the master removes the transaction id the metadata table major compacts, sees the id is missing, so it removes the flags: this is bad the master continues to wait for threads to stop doing work for the transaction the master then sees that there are no references to the tablet and moves it to the failed directory
        Hide
        Keith Turner added a comment - - edited

        A workaround for 1.4.3 and earlier is to remove the !METADATA compaction filter that drops bulk import load flags.

          deleteiter -majc -n bulkLoadFilter -t !METADATA
        

        Then periodically delete bulk load flags that are left by splits. Would need to add the filter back once this bug is fixed.

        I think a solution to fix the bug is to write two things to zookeeper for a bulk import transactions. Delete the first marker in zookeeper when you want work related to the bulk import to stop. Delete the 2nd marker when its safe for major compaction to delete bulk load markers.

        Show
        Keith Turner added a comment - - edited A workaround for 1.4.3 and earlier is to remove the !METADATA compaction filter that drops bulk import load flags. deleteiter -majc -n bulkLoadFilter -t !METADATA Then periodically delete bulk load flags that are left by splits. Would need to add the filter back once this bug is fixed. I think a solution to fix the bug is to write two things to zookeeper for a bulk import transactions. Delete the first marker in zookeeper when you want work related to the bulk import to stop. Delete the 2nd marker when its safe for major compaction to delete bulk load markers.
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk #824 (See https://builds.apache.org/job/Accumulo-Trunk/824/)
        ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466270)
        ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466263)
        ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466204)

        Result = SUCCESS
        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/trunk/src

        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/trunk/src

        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk #824 (See https://builds.apache.org/job/Accumulo-Trunk/824/ ) ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466270) ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466263) ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466204) Result = SUCCESS kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/trunk/src kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/trunk/src ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java /accumulo/trunk/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk-Hadoop-2.0 #182 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/182/)
        ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466270)
        ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466263)
        ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466204)

        Result = SUCCESS
        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/trunk/src

        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/trunk/src

        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk-Hadoop-2.0 #182 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/182/ ) ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466270) ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466263) ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466204) Result = SUCCESS kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/trunk/src kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/trunk/src ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java /accumulo/trunk/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.4.x #289 (See https://builds.apache.org/job/Accumulo-1.4.x/289/)
        ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466259)
        ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466199)

        Result = SUCCESS
        kturner :
        Files :

        • /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java

        ecn :
        Files :

        • /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        • /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java
        • /accumulo/branches/1.4/src/server/src/test/java/org/apache/accumulo/server/zookeeper/TransactionWatcherTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.4.x #289 (See https://builds.apache.org/job/Accumulo-1.4.x/289/ ) ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466259) ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466199) Result = SUCCESS kturner : Files : /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java ecn : Files : /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java /accumulo/branches/1.4/src/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java /accumulo/branches/1.4/src/server/src/test/java/org/apache/accumulo/server/zookeeper/TransactionWatcherTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5 #71 (See https://builds.apache.org/job/Accumulo-1.5/71/)
        ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466269)
        ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466261)
        ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466203)

        Result = SUCCESS
        kturner :
        Files :

        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java

        kturner :
        Files :

        • /accumulo/branches/1.5
        • /accumulo/branches/1.5/assemble
        • /accumulo/branches/1.5/core
        • /accumulo/branches/1.5/examples
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/branches/1.5/server
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/branches/1.5/src

        ecn :
        Files :

        • /accumulo/branches/1.5
        • /accumulo/branches/1.5/assemble
        • /accumulo/branches/1.5/core
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/branches/1.5/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java
        • /accumulo/branches/1.5/server
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java
        • /accumulo/branches/1.5/src
        Show
        Hudson added a comment - Integrated in Accumulo-1.5 #71 (See https://builds.apache.org/job/Accumulo-1.5/71/ ) ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466269) ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466261) ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466203) Result = SUCCESS kturner : Files : /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java kturner : Files : /accumulo/branches/1.5 /accumulo/branches/1.5/assemble /accumulo/branches/1.5/core /accumulo/branches/1.5/examples /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/branches/1.5/server /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/branches/1.5/src ecn : Files : /accumulo/branches/1.5 /accumulo/branches/1.5/assemble /accumulo/branches/1.5/core /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/branches/1.5/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java /accumulo/branches/1.5/server /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java /accumulo/branches/1.5/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5-Hadoop-2.0 #70 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/70/)
        ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466269)
        ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466261)
        ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466203)

        Result = ABORTED
        kturner :
        Files :

        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java

        kturner :
        Files :

        • /accumulo/branches/1.5
        • /accumulo/branches/1.5/assemble
        • /accumulo/branches/1.5/core
        • /accumulo/branches/1.5/examples
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/branches/1.5/server
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/branches/1.5/src

        ecn :
        Files :

        • /accumulo/branches/1.5
        • /accumulo/branches/1.5/assemble
        • /accumulo/branches/1.5/core
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/branches/1.5/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java
        • /accumulo/branches/1.5/server
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java
        • /accumulo/branches/1.5/src
        Show
        Hudson added a comment - Integrated in Accumulo-1.5-Hadoop-2.0 #70 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/70/ ) ACCUMULO-1044 make metadata constraint bulk flag check for column set when new split tablet is added (Revision 1466269) ACCUMULO-1044 fixed bug in bulk load filter (Revision 1466261) ACCUMULO-1044 clearly define "transaction should stop" and "transaction is complete" (Revision 1466203) Result = ABORTED kturner : Files : /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java kturner : Files : /accumulo/branches/1.5 /accumulo/branches/1.5/assemble /accumulo/branches/1.5/core /accumulo/branches/1.5/examples /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/branches/1.5/server /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/branches/1.5/src ecn : Files : /accumulo/branches/1.5 /accumulo/branches/1.5/assemble /accumulo/branches/1.5/core /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/TransactionWatcher.java /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/branches/1.5/fate/src/test/java/org/apache/accumulo/fate/zookeeper/TransactionWatcherTest.java /accumulo/branches/1.5/server /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/master/tableOps/BulkImport.java /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/zookeeper/TransactionWatcher.java /accumulo/branches/1.5/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk-Hadoop-2.0 #183 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/183/)
        ACCUMULO-1044 added unit test for metdata bulk load filter (Revision 1466667)
        ACCUMULO-1044 fixed some issues w/ metadata constraint bulk flag check, made the check more strict, and added a lot test for it (Revision 1466589)

        Result = SUCCESS
        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators
        • /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilterTest.java
        • /accumulo/trunk/src

        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk-Hadoop-2.0 #183 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/183/ ) ACCUMULO-1044 added unit test for metdata bulk load filter (Revision 1466667) ACCUMULO-1044 fixed some issues w/ metadata constraint bulk flag check, made the check more strict, and added a lot test for it (Revision 1466589) Result = SUCCESS kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilterTest.java /accumulo/trunk/src kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java /accumulo/trunk/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5 #72 (See https://builds.apache.org/job/Accumulo-1.5/72/)
        ACCUMULO-1044 added unit test for metdata bulk load filter (Revision 1466664)
        ACCUMULO-1044 fixed some issues w/ metadata constraint bulk flag check, made the check more strict, and added a lot test for it (Revision 1466582)

        Result = UNSTABLE
        kturner :
        Files :

        • /accumulo/branches/1.5
        • /accumulo/branches/1.5/assemble
        • /accumulo/branches/1.5/core
        • /accumulo/branches/1.5/examples
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/branches/1.5/server
        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/iterators
        • /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilterTest.java
        • /accumulo/branches/1.5/src

        kturner :
        Files :

        • /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5 #72 (See https://builds.apache.org/job/Accumulo-1.5/72/ ) ACCUMULO-1044 added unit test for metdata bulk load filter (Revision 1466664) ACCUMULO-1044 fixed some issues w/ metadata constraint bulk flag check, made the check more strict, and added a lot test for it (Revision 1466582) Result = UNSTABLE kturner : Files : /accumulo/branches/1.5 /accumulo/branches/1.5/assemble /accumulo/branches/1.5/core /accumulo/branches/1.5/examples /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/branches/1.5/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/branches/1.5/server /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/iterators /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilterTest.java /accumulo/branches/1.5/src kturner : Files : /accumulo/branches/1.5/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/branches/1.5/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk #825 (See https://builds.apache.org/job/Accumulo-Trunk/825/)
        ACCUMULO-1044 added unit test for metdata bulk load filter (Revision 1466667)
        ACCUMULO-1044 fixed some issues w/ metadata constraint bulk flag check, made the check more strict, and added a lot test for it (Revision 1466589)

        Result = SUCCESS
        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java
        • /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators
        • /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilterTest.java
        • /accumulo/trunk/src

        kturner :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/server
        • /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java
        • /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java
        • /accumulo/trunk/src
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk #825 (See https://builds.apache.org/job/Accumulo-Trunk/825/ ) ACCUMULO-1044 added unit test for metdata bulk load filter (Revision 1466667) ACCUMULO-1044 fixed some issues w/ metadata constraint bulk flag check, made the check more strict, and added a lot test for it (Revision 1466589) Result = SUCCESS kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilter.java /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/iterators/MetadataBulkLoadFilterTest.java /accumulo/trunk/src kturner : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/server /accumulo/trunk/server/src/main/java/org/apache/accumulo/server/constraints/MetadataConstraints.java /accumulo/trunk/server/src/test/java/org/apache/accumulo/server/constraints/MetadataConstraintsTest.java /accumulo/trunk/src
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5-Hadoop-2.0 #86 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/86/)
        ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470632)

        Result = FAILURE
        ecn :
        Files :

        • /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5-Hadoop-2.0 #86 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/86/ ) ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470632) Result = FAILURE ecn : Files : /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5 #87 (See https://builds.apache.org/job/Accumulo-1.5/87/)
        ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470632)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5 #87 (See https://builds.apache.org/job/Accumulo-1.5/87/ ) ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470632) Result = SUCCESS ecn : Files : /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk-Hadoop-2.0 #198 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/198/)
        ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470633)

        Result = FAILURE
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/pom.xml
        • /accumulo/trunk/server
        • /accumulo/trunk/src
        • /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk-Hadoop-2.0 #198 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/198/ ) ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470633) Result = FAILURE ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/pom.xml /accumulo/trunk/server /accumulo/trunk/src /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk #840 (See https://builds.apache.org/job/Accumulo-Trunk/840/)
        ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470633)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/pom.xml
        • /accumulo/trunk/server
        • /accumulo/trunk/src
        • /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk #840 (See https://builds.apache.org/job/Accumulo-Trunk/840/ ) ACCUMULO-1044 fix functional test to handle new constraint (Revision 1470633) Result = SUCCESS ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/pom.xml /accumulo/trunk/server /accumulo/trunk/src /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5-Hadoop-2.0 #87 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/87/)
        ACCUMULO-1044 fix unused import (Revision 1470965)

        Result = FAILURE
        ecn :
        Files :

        • /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5-Hadoop-2.0 #87 (See https://builds.apache.org/job/Accumulo-1.5-Hadoop-2.0/87/ ) ACCUMULO-1044 fix unused import (Revision 1470965) Result = FAILURE ecn : Files : /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk #841 (See https://builds.apache.org/job/Accumulo-Trunk/841/)
        ACCUMULO-1044 fix unused import (Revision 1470966)

        Result = FAILURE
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/pom.xml
        • /accumulo/trunk/server
        • /accumulo/trunk/src
        • /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk #841 (See https://builds.apache.org/job/Accumulo-Trunk/841/ ) ACCUMULO-1044 fix unused import (Revision 1470966) Result = FAILURE ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/pom.xml /accumulo/trunk/server /accumulo/trunk/src /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-1.5 #88 (See https://builds.apache.org/job/Accumulo-1.5/88/)
        ACCUMULO-1044 fix unused import (Revision 1470965)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-1.5 #88 (See https://builds.apache.org/job/Accumulo-1.5/88/ ) ACCUMULO-1044 fix unused import (Revision 1470965) Result = SUCCESS ecn : Files : /accumulo/branches/1.5/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Hudson added a comment -

        Integrated in Accumulo-Trunk-Hadoop-2.0 #199 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/199/)
        ACCUMULO-1044 fix unused import (Revision 1470966)

        Result = SUCCESS
        ecn :
        Files :

        • /accumulo/trunk
        • /accumulo/trunk/assemble
        • /accumulo/trunk/core
        • /accumulo/trunk/examples
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java
        • /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java
        • /accumulo/trunk/pom.xml
        • /accumulo/trunk/server
        • /accumulo/trunk/src
        • /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Show
        Hudson added a comment - Integrated in Accumulo-Trunk-Hadoop-2.0 #199 (See https://builds.apache.org/job/Accumulo-Trunk-Hadoop-2.0/199/ ) ACCUMULO-1044 fix unused import (Revision 1470966) Result = SUCCESS ecn : Files : /accumulo/trunk /accumulo/trunk/assemble /accumulo/trunk/core /accumulo/trunk/examples /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/ZooStore.java /accumulo/trunk/fate/src/main/java/org/apache/accumulo/fate/zookeeper/ZooSession.java /accumulo/trunk/pom.xml /accumulo/trunk/server /accumulo/trunk/src /accumulo/trunk/test/src/main/java/org/apache/accumulo/test/functional/SplitRecoveryTest.java
        Hide
        Eric Newton added a comment - - edited

        So, in 1.5.0, the situation is much better, but there's still a race condition:

        • master delegates loading to a tserver worker
        • worker tserver assigns the files to the appropriate serving tservers
        • the serving tserver fails to load the file due to contention, full gc, and other distributed computing problems; worker tserver times out
        • worker tserver gives up, reports failure to the master
        • a thread on the serving tserver wakes up and starts the tablet load
        • master talks to all the worker servers, and finds them not working on the transaction
        • master looks for references, finds none, and moves the file to failures
        • the server tserver awakes up again and finishes the file load and updates the !METADATA table
        • the master wipes out any loaded markers

        The master needs to ask the serving tservers if they are done with the transaction; and since the master doesn't know which server got the files, it needs to ask all of them. The serving tservers will then be up-to-date with zookeeper, and will never load the file afterwards. If the master asks the serving tserver to remove the loaded flags, then we do not need to be concerned about splits.

        Show
        Eric Newton added a comment - - edited So, in 1.5.0, the situation is much better, but there's still a race condition: master delegates loading to a tserver worker worker tserver assigns the files to the appropriate serving tservers the serving tserver fails to load the file due to contention, full gc, and other distributed computing problems; worker tserver times out worker tserver gives up, reports failure to the master a thread on the serving tserver wakes up and starts the tablet load master talks to all the worker servers, and finds them not working on the transaction master looks for references, finds none, and moves the file to failures the server tserver awakes up again and finishes the file load and updates the !METADATA table the master wipes out any loaded markers The master needs to ask the serving tservers if they are done with the transaction; and since the master doesn't know which server got the files, it needs to ask all of them. The serving tservers will then be up-to-date with zookeeper, and will never load the file afterwards. If the master asks the serving tserver to remove the loaded flags, then we do not need to be concerned about splits.
        Hide
        Eric Newton added a comment -

        In a private discussion with Keith, we've determined that this is no longer a problem: the master asks all tservers if they are working on a transaction, which synch's them to zookeeper and the non-existence of the transaction.

        Show
        Eric Newton added a comment - In a private discussion with Keith, we've determined that this is no longer a problem: the master asks all tservers if they are working on a transaction, which synch's them to zookeeper and the non-existence of the transaction.
        Hide
        ASF subversion and git services added a comment -

        Commit ff226a78995057518ac354e671d90f1bb4c30884 in branch refs/heads/master from Eric Newton
        [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=ff226a7 ]

        ACCUMULO-1605

        Added many finally clauses to close spans in the presence of exceptions.

        I suspect that the extreme depth of the traces is caused by traces that
        are started and never stopped. This most likely happens in the presence
        of errors, where exceptions jump around the the span.stop calls.
        In particular, when a file is missing for a compaction. This is known
        to occur in 1.4 when bulk imports fail. See ACCUMULO-1044.

        Show
        ASF subversion and git services added a comment - Commit ff226a78995057518ac354e671d90f1bb4c30884 in branch refs/heads/master from Eric Newton [ https://git-wip-us.apache.org/repos/asf?p=accumulo.git;h=ff226a7 ] ACCUMULO-1605 Added many finally clauses to close spans in the presence of exceptions. I suspect that the extreme depth of the traces is caused by traces that are started and never stopped. This most likely happens in the presence of errors, where exceptions jump around the the span.stop calls. In particular, when a file is missing for a compaction. This is known to occur in 1.4 when bulk imports fail. See ACCUMULO-1044 .

          People

          • Assignee:
            Eric Newton
            Reporter:
            Eric Newton
          • Votes:
            0 Vote for this issue
            Watchers:
            4 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development