Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.98.0, 0.94.7, 0.95.1
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Incompatible change, Reviewed
    • Release Note:
      Hide
      In order to avoid name collision with hdfs snapshot, ".snapshot" under HBase root directory must be renamed as ".hbase-snapshot" before deployment of 0.94.7 or later release is started.

      Don't do rolling upgrade between 0.94.6.1 and 0.94.7 if there're on-going snapshots activities.
      Sequence of actions should be:

      1. make sure all on-going snapshots activities have completed
      2. shutdown all the masters
      3. rename ".snapshot" under HBase root directory as ".hbase-snapshot"
      4. upgrade the masters to 0.94.7+ release (You now can perform clone or restore, but not taking snapshots)
      5. upgrade the region servers
      (You should not take snapshots until all the regions server are aligned to 0.94.7+ release)

      If you take a snapshot during the RS upgrades nothing critical happens. The ".snapshot" folder would appear again and the snapshot will fail since half of the data will be in ".hbase-snapshot" and the other half in ".snapshot"
      Show
      In order to avoid name collision with hdfs snapshot, ".snapshot" under HBase root directory must be renamed as ".hbase-snapshot" before deployment of 0.94.7 or later release is started. Don't do rolling upgrade between 0.94.6.1 and 0.94.7 if there're on-going snapshots activities. Sequence of actions should be: 1. make sure all on-going snapshots activities have completed 2. shutdown all the masters 3. rename ".snapshot" under HBase root directory as ".hbase-snapshot" 4. upgrade the masters to 0.94.7+ release (You now can perform clone or restore, but not taking snapshots) 5. upgrade the region servers (You should not take snapshots until all the regions server are aligned to 0.94.7+ release) If you take a snapshot during the RS upgrades nothing critical happens. The ".snapshot" folder would appear again and the snapshot will fail since half of the data will be in ".hbase-snapshot" and the other half in ".snapshot"

      Description

      Testing HBase Snapshot on top of Hadoop's Snapshot branch (http://svn.apache.org/viewvc/hadoop/common/branches/HDFS-2802/), we found that both features used '.snapshot' directory to store metadata.

      HDFS (built from HDFS-2802 branch) doesn't allow paths with .snapshot as a component

      From discussion on dev@hbase.apache.org, (see http://search-hadoop.com/m/kY6C3cXMs51), consensus was to rename '.snapshot' directory in HBase so that both features can co-exist smoothly.

      1. 8352-0.94-v4.txt
        10 kB
        Ted Yu
      2. 8352-trunk-v6.txt
        10 kB
        Ted Yu
      3. 8352-0.94-v3.txt
        10 kB
        Ted Yu
      4. 8352-trunk-v5.txt
        9 kB
        Ted Yu
      5. 8352-trunk-v4.txt
        10 kB
        Ted Yu
      6. 8352-trunk-v3.txt
        7 kB
        Ted Yu
      7. 8352-0.94-v2.txt
        8 kB
        Ted Yu
      8. 8352-trunk-v2.txt
        7 kB
        Ted Yu
      9. 8352-trunk.txt
        7 kB
        Ted Yu
      10. 8352-0.94-v1.txt
        7 kB
        Ted Yu

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          Thanks for everyone. You guys have done a great job!

          Show
          Tsz Wo Nicholas Sze added a comment - Thanks for everyone. You guys have done a great job!
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94-security #138 (See https://builds.apache.org/job/HBase-0.94-security/138/)
          HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468671)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotsFromAdmin.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Show
          Hudson added a comment - Integrated in HBase-0.94-security #138 (See https://builds.apache.org/job/HBase-0.94-security/138/ ) HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468671) Result = FAILURE tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotsFromAdmin.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #501 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/501/)
          HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468669)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java
          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK-on-Hadoop-2.0.0 #501 (See https://builds.apache.org/job/HBase-TRUNK-on-Hadoop-2.0.0/501/ ) HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468669) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Hide
          Hudson added a comment -

          Integrated in HBase-TRUNK #4067 (See https://builds.apache.org/job/HBase-TRUNK/4067/)
          HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468669)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java
          • /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
          • /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
          • /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Show
          Hudson added a comment - Integrated in HBase-TRUNK #4067 (See https://builds.apache.org/job/HBase-TRUNK/4067/ ) HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468669) Result = FAILURE tedyu : Files : /hbase/trunk/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java /hbase/trunk/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java /hbase/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java /hbase/trunk/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Hide
          Hudson added a comment -

          Integrated in HBase-0.94 #962 (See https://builds.apache.org/job/HBase-0.94/962/)
          HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468671)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java
          • /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotsFromAdmin.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
          • /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Show
          Hudson added a comment - Integrated in HBase-0.94 #962 (See https://builds.apache.org/job/HBase-0.94/962/ ) HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468671) Result = SUCCESS tedyu : Files : /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/HConstants.java /hbase/branches/0.94/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotsFromAdmin.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java /hbase/branches/0.94/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Hide
          Hudson added a comment -

          Integrated in hbase-0.95-on-hadoop2 #71 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/71/)
          HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468670)

          Result = FAILURE
          tedyu :
          Files :

          • /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java
          • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
          • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
          • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
          • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Show
          Hudson added a comment - Integrated in hbase-0.95-on-hadoop2 #71 (See https://builds.apache.org/job/hbase-0.95-on-hadoop2/71/ ) HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468670) Result = FAILURE tedyu : Files : /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Hide
          Hudson added a comment -

          Integrated in hbase-0.95 #150 (See https://builds.apache.org/job/hbase-0.95/150/)
          HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468670)

          Result = SUCCESS
          tedyu :
          Files :

          • /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java
          • /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java
          • /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java
          • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java
          • /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Show
          Hudson added a comment - Integrated in hbase-0.95 #150 (See https://builds.apache.org/job/hbase-0.95/150/ ) HBASE-8352 Rename '.snapshot' directory (Ted Yu) (Revision 1468670) Result = SUCCESS tedyu : Files : /hbase/branches/0.95/hbase-client/src/test/java/org/apache/hadoop/hbase/client/TestSnapshotFromAdmin.java /hbase/branches/0.95/hbase-common/src/main/java/org/apache/hadoop/hbase/HConstants.java /hbase/branches/0.95/hbase-server/src/main/java/org/apache/hadoop/hbase/master/snapshot/SnapshotManager.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestExportSnapshot.java /hbase/branches/0.95/hbase-server/src/test/java/org/apache/hadoop/hbase/snapshot/TestSnapshotDescriptionUtils.java
          Hide
          Ted Yu added a comment -

          Integrated to 0.94, 0.95 and trunk.

          Thanks for the reviews, Jon, Matteo, Lars and Andy.

          Show
          Ted Yu added a comment - Integrated to 0.94, 0.95 and trunk. Thanks for the reviews, Jon, Matteo, Lars and Andy.
          Hide
          Ted Yu added a comment -

          @Matteo:
          Were you suggesting modification based on patch v6 ?

          Show
          Ted Yu added a comment - @Matteo: Were you suggesting modification based on patch v6 ?
          Hide
          Matteo Bertozzi added a comment -

          if the inital folder name was ".hbase-snapshot" today this constant probably it will be called HDFS_SNAPSHOTS_DIR_NAME, since the idea of reserving it is because that name is taken by hdfs...

          otherwise we'll end up in .98 to look at the code and say... hey this OLD_SNAPSHOT is no longer used... we can remove it.

          Show
          Matteo Bertozzi added a comment - if the inital folder name was ".hbase-snapshot" today this constant probably it will be called HDFS_SNAPSHOTS_DIR_NAME, since the idea of reserving it is because that name is taken by hdfs... otherwise we'll end up in .98 to look at the code and say... hey this OLD_SNAPSHOT is no longer used... we can remove it.
          Hide
          Andrew Purtell added a comment -

          OLD_SNAPSHOT_DIR_NAME lgtm

          Show
          Andrew Purtell added a comment - OLD_SNAPSHOT_DIR_NAME lgtm
          Hide
          Ted Yu added a comment -

          I thought about that. getCompletedSnapshots() references ".snapshot" (old name).
          I can expose the following constant and reference it from getCompletedSnapshots():

            /* Name of old snapshot directory. See HBASE-8352 for details on why it needs to be renamed */
            public static final String OLD_SNAPSHOT_DIR_NAME = ".snapshot";
          
          Show
          Ted Yu added a comment - I thought about that. getCompletedSnapshots() references ".snapshot" (old name). I can expose the following constant and reference it from getCompletedSnapshots(): /* Name of old snapshot directory. See HBASE-8352 for details on why it needs to be renamed */ public static final String OLD_SNAPSHOT_DIR_NAME = ".snapshot" ;
          Hide
          Andrew Purtell added a comment -

          Minor nit on patch v5, can be fixed up on commit, let's put ".snapshot" on the non-table-dirs list using a constant defined somewhere with javadoc as to why.

          Show
          Andrew Purtell added a comment - Minor nit on patch v5, can be fixed up on commit, let's put ".snapshot" on the non-table-dirs list using a constant defined somewhere with javadoc as to why.
          Hide
          Ted Yu added a comment -

          Patch v3 for 0.94 aligns with patch v5 for trunk.

          Running org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient
          2013-04-16 14:08:41.240 java[3877:1203] Unable to load realm info from SCDynamicStore
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.711 sec
          Running org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient
          2013-04-16 14:10:42.576 java[3902:1203] Unable to load realm info from SCDynamicStore
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 142.78 sec
          Running org.apache.hadoop.hbase.client.TestSnapshotFromAdmin
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.289 sec
          Running org.apache.hadoop.hbase.client.TestSnapshotFromClient
          2013-04-16 14:13:10.506 java[3926:1203] Unable to load realm info from SCDynamicStore
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.47 sec
          Running org.apache.hadoop.hbase.client.TestSnapshotsFromAdmin
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.223 sec
          Running org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster
          2013-04-16 14:13:53.041 java[3944:1203] Unable to load realm info from SCDynamicStore
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.12 sec
          Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache
          2013-04-16 14:14:36.754 java[3963:1203] Unable to load realm info from SCDynamicStore
          Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.994 sec
          Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotHFileCleaner
          2013-04-16 14:15:30.314 java[3974:1203] Unable to load realm info from SCDynamicStore
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.506 sec
          Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotLogCleaner
          2013-04-16 14:15:31.065 java[3976:1203] Unable to load realm info from SCDynamicStore
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.479 sec
          Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotManager
          2013-04-16 14:15:31.887 java[3978:1203] Unable to load realm info from SCDynamicStore
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.723 sec
          Running org.apache.hadoop.hbase.snapshot.TestExportSnapshot
          2013-04-16 14:15:32.888 java[3980:1203] Unable to load realm info from SCDynamicStore
          Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.427 sec
          Running org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient
          2013-04-16 14:16:03.943 java[4007:1203] Unable to load realm info from SCDynamicStore
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.386 sec
          Running org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient
          2013-04-16 14:16:49.908 java[4028:1203] Unable to load realm info from SCDynamicStore
          Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 108.129 sec
          Running org.apache.hadoop.hbase.snapshot.TestRestoreSnapshotHelper
          2013-04-16 14:18:38.512 java[4049:1203] Unable to load realm info from SCDynamicStore
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.839 sec
          Running org.apache.hadoop.hbase.snapshot.TestSnapshotDescriptionUtils
          2013-04-16 14:18:39.561 java[4051:1203] Unable to load realm info from SCDynamicStore
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.474 sec
          Running org.apache.hadoop.hbase.snapshot.TestSnapshotLogSplitter
          2013-04-16 14:18:40.295 java[4053:1203] Unable to load realm info from SCDynamicStore
          Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.012 sec
          Running org.apache.hadoop.hbase.snapshot.TestSnapshotTask
          Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.24 sec
          
          Show
          Ted Yu added a comment - Patch v3 for 0.94 aligns with patch v5 for trunk. Running org.apache.hadoop.hbase.client.TestCloneSnapshotFromClient 2013-04-16 14:08:41.240 java[3877:1203] Unable to load realm info from SCDynamicStore Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 120.711 sec Running org.apache.hadoop.hbase.client.TestRestoreSnapshotFromClient 2013-04-16 14:10:42.576 java[3902:1203] Unable to load realm info from SCDynamicStore Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 142.78 sec Running org.apache.hadoop.hbase.client.TestSnapshotFromAdmin Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.289 sec Running org.apache.hadoop.hbase.client.TestSnapshotFromClient 2013-04-16 14:13:10.506 java[3926:1203] Unable to load realm info from SCDynamicStore Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 37.47 sec Running org.apache.hadoop.hbase.client.TestSnapshotsFromAdmin Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 4.223 sec Running org.apache.hadoop.hbase.master.cleaner.TestSnapshotFromMaster 2013-04-16 14:13:53.041 java[3944:1203] Unable to load realm info from SCDynamicStore Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 43.12 sec Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotFileCache 2013-04-16 14:14:36.754 java[3963:1203] Unable to load realm info from SCDynamicStore Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 2.994 sec Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotHFileCleaner 2013-04-16 14:15:30.314 java[3974:1203] Unable to load realm info from SCDynamicStore Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.506 sec Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotLogCleaner 2013-04-16 14:15:31.065 java[3976:1203] Unable to load realm info from SCDynamicStore Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.479 sec Running org.apache.hadoop.hbase.master.snapshot.TestSnapshotManager 2013-04-16 14:15:31.887 java[3978:1203] Unable to load realm info from SCDynamicStore Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.723 sec Running org.apache.hadoop.hbase.snapshot.TestExportSnapshot 2013-04-16 14:15:32.888 java[3980:1203] Unable to load realm info from SCDynamicStore Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 30.427 sec Running org.apache.hadoop.hbase.snapshot.TestFlushSnapshotFromClient 2013-04-16 14:16:03.943 java[4007:1203] Unable to load realm info from SCDynamicStore Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 45.386 sec Running org.apache.hadoop.hbase.snapshot.TestRestoreFlushSnapshotFromClient 2013-04-16 14:16:49.908 java[4028:1203] Unable to load realm info from SCDynamicStore Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 108.129 sec Running org.apache.hadoop.hbase.snapshot.TestRestoreSnapshotHelper 2013-04-16 14:18:38.512 java[4049:1203] Unable to load realm info from SCDynamicStore Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.839 sec Running org.apache.hadoop.hbase.snapshot.TestSnapshotDescriptionUtils 2013-04-16 14:18:39.561 java[4051:1203] Unable to load realm info from SCDynamicStore Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.474 sec Running org.apache.hadoop.hbase.snapshot.TestSnapshotLogSplitter 2013-04-16 14:18:40.295 java[4053:1203] Unable to load realm info from SCDynamicStore Tests run: 2, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.012 sec Running org.apache.hadoop.hbase.snapshot.TestSnapshotTask Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.24 sec
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12579005/8352-trunk-v5.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12579005/8352-trunk-v5.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 9 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5330//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          +1 from me as well.

          Show
          Lars Hofhansl added a comment - +1 from me as well.
          Hide
          Jesse Yates added a comment -

          +1 lgtm. Thanks Ted!

          Show
          Jesse Yates added a comment - +1 lgtm. Thanks Ted!
          Hide
          Matteo Bertozzi added a comment -

          +1 v5 looks good to me

          Show
          Matteo Bertozzi added a comment - +1 v5 looks good to me
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12578998/8352-trunk-v4.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12578998/8352-trunk-v4.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 9 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5329//console This message is automatically generated.
          Hide
          Jonathan Hsieh added a comment -

          lgtm +1.

          Show
          Jonathan Hsieh added a comment - lgtm +1.
          Hide
          Ted Yu added a comment -

          Patch v5 addresses Jon's comment

          Show
          Ted Yu added a comment - Patch v5 addresses Jon's comment
          Hide
          Jonathan Hsieh added a comment - - edited

          nit: Prefer keeping the no argument version – this extra generally never used arg makes this harder to read.

          -      List<SnapshotDescription> snapshots = snapshotManager.getCompletedSnapshots();
          +      List<SnapshotDescription> snapshots = snapshotManager.getCompletedSnapshots(null);
          

          nit: prefer if you create a private more generic version with the path argument but keep the older no argument version public. the current way exposes possibilities that don't need to be.

          So instead of this:

             /**
              * Gets the list of all completed snapshots.
          +   * @param snapshotDir snapshot directory
              * @return list of SnapshotDescriptions
              * @throws IOException File system exception
              */
          -  public List<SnapshotDescription> getCompletedSnapshots() throws IOException {
          +  public List<SnapshotDescription> getCompletedSnapshots(Path snapshotDir) throws IOException {
               List<SnapshotDescription> snapshotDescs = new ArrayList<SnapshotDescription>();
               // first create the snapshot root path and check to see if it exists
          -    Path snapshotDir = SnapshotDescriptionUtils.getSnapshotsDir(rootDir);
               FileSystem fs = master.getMasterFileSystem().getFileSystem();
          +    if (snapshotDir == null) snapshotDir = SnapshotDescriptionUtils.getSnapshotsDir(rootDir);
           
               // if there are no snapshots, return an empty list
               if (!fs.exists(snapshotDir)) {
          @@ -877,6 +878,15 @@
          

          have something more like this:

            public List<SnapshotDescription> getCompletedSnapshots() throws IOException { 
              return getCompletedSnapshots(SnapshotDescriptionUtils.getSnapshotsDir(rootDir));
            }
            private List<SnapshotDescription> getCompletedSnapshots(Path snapshotDir) throws IOException {
              ...
            }
          

          Otherwise, lgtm.

          Show
          Jonathan Hsieh added a comment - - edited nit: Prefer keeping the no argument version – this extra generally never used arg makes this harder to read. - List<SnapshotDescription> snapshots = snapshotManager.getCompletedSnapshots(); + List<SnapshotDescription> snapshots = snapshotManager.getCompletedSnapshots( null ); nit: prefer if you create a private more generic version with the path argument but keep the older no argument version public. the current way exposes possibilities that don't need to be. So instead of this: /** * Gets the list of all completed snapshots. + * @param snapshotDir snapshot directory * @ return list of SnapshotDescriptions * @ throws IOException File system exception */ - public List<SnapshotDescription> getCompletedSnapshots() throws IOException { + public List<SnapshotDescription> getCompletedSnapshots(Path snapshotDir) throws IOException { List<SnapshotDescription> snapshotDescs = new ArrayList<SnapshotDescription>(); // first create the snapshot root path and check to see if it exists - Path snapshotDir = SnapshotDescriptionUtils.getSnapshotsDir(rootDir); FileSystem fs = master.getMasterFileSystem().getFileSystem(); + if (snapshotDir == null ) snapshotDir = SnapshotDescriptionUtils.getSnapshotsDir(rootDir); // if there are no snapshots, return an empty list if (!fs.exists(snapshotDir)) { @@ -877,6 +878,15 @@ have something more like this: public List<SnapshotDescription> getCompletedSnapshots() throws IOException { return getCompletedSnapshots(SnapshotDescriptionUtils.getSnapshotsDir(rootDir)); } private List<SnapshotDescription> getCompletedSnapshots(Path snapshotDir) throws IOException { ... } Otherwise, lgtm.
          Hide
          Ted Yu added a comment -

          Patch v4 uses oldSnapshotDir in checkSnapshotSupport().

          Show
          Ted Yu added a comment - Patch v4 uses oldSnapshotDir in checkSnapshotSupport().
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12578983/8352-trunk-v3.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.master.snapshot.TestSnapshotManager

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12578983/8352-trunk-v3.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 9 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.master.snapshot.TestSnapshotManager Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5327//console This message is automatically generated.
          Hide
          Lars Hofhansl added a comment -

          This does not seem right:

          +    // check if an older version of snapshot directory was present
          +    Path oldSnapshotDir = new Path(mfs.getRootDir(), ".snapshot");
          +    FileSystem fs = mfs.getFileSystem();
          +    List<SnapshotDescription> ss = getCompletedSnapshots();
          +    if (ss != null && !ss.isEmpty()) {
          +      LOG.error("Snapshots from an earlier release were found under: " + oldSnapshotDir);
          +      LOG.error("Please rename the directory as " + HConstants.SNAPSHOT_DIR_NAME);
          +    }
          

          If never used the old snapshot dir for anything and will just return the new snapshots anyway.

          Show
          Lars Hofhansl added a comment - This does not seem right: + // check if an older version of snapshot directory was present + Path oldSnapshotDir = new Path(mfs.getRootDir(), ".snapshot" ); + FileSystem fs = mfs.getFileSystem(); + List<SnapshotDescription> ss = getCompletedSnapshots(); + if (ss != null && !ss.isEmpty()) { + LOG.error( "Snapshots from an earlier release were found under: " + oldSnapshotDir); + LOG.error( "Please rename the directory as " + HConstants.SNAPSHOT_DIR_NAME); + } If never used the old snapshot dir for anything and will just return the new snapshots anyway.
          Hide
          Ted Yu added a comment -

          See if patch v3 addresses Jon's comment.

          Show
          Ted Yu added a comment - See if patch v3 addresses Jon's comment.
          Hide
          Hadoop QA added a comment -

          +1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12578966/8352-trunk-v2.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          +1 core tests. The patch passed unit tests in .

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - +1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12578966/8352-trunk-v2.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 9 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. +1 core tests . The patch passed unit tests in . Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5325//console This message is automatically generated.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          > If 2.0.5-alpha is the underlying hdfs, I think accessing .snapshot dir would not be allowed.

          I thought hdfs snapshots dirs are readable and browseable just like normal dirs. Only direct write operations would be blocked. Is this incorrect?

          Jonathan, you are right that snapshot dirs are readable and browseable. Let me clarify this:

          With snapshot feature, dirs can be set as snapshottable and snapshots can only be taken on snapshottable dirs. Once a dir (say /foo) is snapshottable, ".snapshot" is used for accessing snapshot and the usual API/CLI will work for it, e.g.

          • show all snapshots of foo
            fs -ls /foo/.snapshot   
            
          • list the file in snapshot s0
            fs -ls /foo/.snapshot/s0
            
          • copy a file inside s0 to /tmp
            fs -cp /foo/.snapshot/s0/bar.txt /tmp
            

          For non-snapshottable dir (say /bar), /bar/.snapshot is an non-existing file. That's why Ted got file not found from CLI.

          Show
          Tsz Wo Nicholas Sze added a comment - > If 2.0.5-alpha is the underlying hdfs, I think accessing .snapshot dir would not be allowed. I thought hdfs snapshots dirs are readable and browseable just like normal dirs. Only direct write operations would be blocked. Is this incorrect? Jonathan, you are right that snapshot dirs are readable and browseable. Let me clarify this: With snapshot feature, dirs can be set as snapshottable and snapshots can only be taken on snapshottable dirs. Once a dir (say /foo) is snapshottable, ".snapshot" is used for accessing snapshot and the usual API/CLI will work for it, e.g. show all snapshots of foo fs -ls /foo/.snapshot list the file in snapshot s0 fs -ls /foo/.snapshot/s0 copy a file inside s0 to /tmp fs -cp /foo/.snapshot/s0/bar.txt /tmp For non-snapshottable dir (say /bar), /bar/.snapshot is an non-existing file. That's why Ted got file not found from CLI.
          Hide
          Enis Soztutar added a comment -

          Release notes seems good. There is a upgrade path from 0.94.6 to 0.96, which is not addressed. We can add to the upgrade section in the book about this as well.

          Show
          Enis Soztutar added a comment - Release notes seems good. There is a upgrade path from 0.94.6 to 0.96, which is not addressed. We can add to the upgrade section in the book about this as well.
          Hide
          Jonathan Hsieh added a comment -

          The insertion point in the code is correct but the fs.exists(oldSnapshotDir) check is not sufficient – if hdfs snapshots is on, there will potentially be a .snapshot dir "exists" but that corresponds to the hdfs snapshot. We need to go into the .snapshot dir and see if there are any .snapshotinfo files present one directory level deeper.

          See dir structure of a snapshot in https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java#L38

          Show
          Jonathan Hsieh added a comment - The insertion point in the code is correct but the fs.exists(oldSnapshotDir) check is not sufficient – if hdfs snapshots is on, there will potentially be a .snapshot dir "exists" but that corresponds to the hdfs snapshot. We need to go into the .snapshot dir and see if there are any .snapshotinfo files present one directory level deeper. See dir structure of a snapshot in https://github.com/apache/hbase/blob/trunk/hbase-server/src/main/java/org/apache/hadoop/hbase/snapshot/SnapshotDescriptionUtils.java#L38
          Hide
          Ted Yu added a comment -

          Patch v2 has the following in checkSnapshotSupport():

              if (fs.exists(oldSnapshotDir)) {
                LOG.error("Snapshots from an earlier release were found under: " + oldSnapshotDir);
                LOG.error("Please rename the directory as " + HConstants.SNAPSHOT_DIR_NAME);
              }
          

          What additional warning should be added ?

          Show
          Ted Yu added a comment - Patch v2 has the following in checkSnapshotSupport(): if (fs.exists(oldSnapshotDir)) { LOG.error( "Snapshots from an earlier release were found under: " + oldSnapshotDir); LOG.error( "Please rename the directory as " + HConstants.SNAPSHOT_DIR_NAME); } What additional warning should be added ?
          Hide
          Jonathan Hsieh added a comment -

          So you're good with the patch, Jon?

          Let's add a warning to logs, and then I'm fine with the approach.

          Show
          Jonathan Hsieh added a comment - So you're good with the patch, Jon? Let's add a warning to logs, and then I'm fine with the approach.
          Hide
          Lars Hofhansl added a comment -

          So you're good with the patch, Jon?

          Show
          Lars Hofhansl added a comment - So you're good with the patch, Jon?
          Hide
          Jonathan Hsieh added a comment -

          I'm suggesting that for future 0.94's (0.94.7 let say) we print a warning if the original directory is used. We decide to warn if hbase detects the .snapshot dir is being used to store hbase table snapshot metadata.

          See release notes in HDFS-4666 for how ".snapshot" is handled by hdfs with snapshot capability.

          The release note doesn't say anything that would prevent the ls and potential read operations necessary to check and log.

          HDFS-4666's release note: "This is an INCOMPATIBLE CHANGE: ".snapshot" becomes a reserved inode name so that users cannot create a file/directory with ".snapshot" as the name. If ".snapshot" is used in a previous version of HDFS, it must be renamed before upgrade; otherwise, upgrade will fail."

          HDFS upgrades failing because of existing .snapshot dirs is great.

          From HDFS snapshot design doc / admin usage: https://issues.apache.org/jira/secure/attachment/12551474/Snapshots20121030.pdf

          Deleting a snapshot:
          > rm -r <snapshottable_dir_path>/.snapshot/snapname
          Listing all the snapshots:
          > ls <snapshottable_dir_path>/.snapshot/
          

          We can ls the .snapshot dir as if it were a normal dir.

          Show
          Jonathan Hsieh added a comment - I'm suggesting that for future 0.94's (0.94.7 let say) we print a warning if the original directory is used. We decide to warn if hbase detects the .snapshot dir is being used to store hbase table snapshot metadata. See release notes in HDFS-4666 for how ".snapshot" is handled by hdfs with snapshot capability. The release note doesn't say anything that would prevent the ls and potential read operations necessary to check and log. HDFS-4666 's release note: "This is an INCOMPATIBLE CHANGE: ".snapshot" becomes a reserved inode name so that users cannot create a file/directory with ".snapshot" as the name. If ".snapshot" is used in a previous version of HDFS, it must be renamed before upgrade; otherwise, upgrade will fail." HDFS upgrades failing because of existing .snapshot dirs is great. From HDFS snapshot design doc / admin usage: https://issues.apache.org/jira/secure/attachment/12551474/Snapshots20121030.pdf Deleting a snapshot: > rm -r <snapshottable_dir_path>/.snapshot/snapname Listing all the snapshots: > ls <snapshottable_dir_path>/.snapshot/ We can ls the .snapshot dir as if it were a normal dir.
          Hide
          Lars Hofhansl added a comment -

          Release notes look good. +1 on v2.

          Show
          Lars Hofhansl added a comment - Release notes look good. +1 on v2.
          Hide
          Ted Yu added a comment -

          See release notes in HDFS-4666 for how ".snapshot" is handled by hdfs with snapshot capability.

          Show
          Ted Yu added a comment - See release notes in HDFS-4666 for how ".snapshot" is handled by hdfs with snapshot capability.
          Hide
          Jonathan Hsieh added a comment -

          If 2.0.5-alpha is the underlying hdfs, I think accessing .snapshot dir would not be allowed.

          I thought hdfs snapshots dirs are readable and browseable just like normal dirs. Only direct write operations would be blocked. Is this incorrect?

          w.r.t. Lars' suggestion of introducing 'hbase.snapshot.use_newdir' config parameter, I think we'd better keep logic simple - considering HBase snapshots usage is in very early stage.

          I agree with keeping it simple, but I have to insist that dealing with compatibility is one of the consequences of backporting new features to stable releases. Early or not it has been released in a stable apache hbase version.

          Show
          Jonathan Hsieh added a comment - If 2.0.5-alpha is the underlying hdfs, I think accessing .snapshot dir would not be allowed. I thought hdfs snapshots dirs are readable and browseable just like normal dirs. Only direct write operations would be blocked. Is this incorrect? w.r.t. Lars' suggestion of introducing 'hbase.snapshot.use_newdir' config parameter, I think we'd better keep logic simple - considering HBase snapshots usage is in very early stage. I agree with keeping it simple, but I have to insist that dealing with compatibility is one of the consequences of backporting new features to stable releases. Early or not it has been released in a stable apache hbase version.
          Hide
          Ted Yu added a comment -

          Trunk patch v2 adds ".snapshot" to HBASE_NON_TABLE_DIRS

          Show
          Ted Yu added a comment - Trunk patch v2 adds ".snapshot" to HBASE_NON_TABLE_DIRS
          Hide
          Jonathan Hsieh added a comment - - edited

          If hbase.snapshot.use_newdir is true we use .hbase_snapshot, if false we use .snapshot

          Since HDFS snapshots seems to have an api to enable it, we could query for that instead of introducing a new config. We could also do a hadoop version check type query (if 1.x use old default, if 2.x use new).

          We'd add both .snapshot and .hbase_snapshot to HBASE_NON_TABLE_DIRS

          +1 to this in 0.94+. Regardless of what fix we choose, preemptively doing this doesn't hurt.

          Thoughts? Too much effort?

          I think if we put the warnings that show up in logs in, I'll be satisfied.

          Show
          Jonathan Hsieh added a comment - - edited If hbase.snapshot.use_newdir is true we use .hbase_snapshot, if false we use .snapshot Since HDFS snapshots seems to have an api to enable it, we could query for that instead of introducing a new config. We could also do a hadoop version check type query (if 1.x use old default, if 2.x use new). We'd add both .snapshot and .hbase_snapshot to HBASE_NON_TABLE_DIRS +1 to this in 0.94+. Regardless of what fix we choose, preemptively doing this doesn't hurt. Thoughts? Too much effort? I think if we put the warnings that show up in logs in, I'll be satisfied.
          Hide
          Ted Yu added a comment -

          w.r.t. Lars' suggestion of introducing 'hbase.snapshot.use_newdir' config parameter, I think we'd better keep logic simple - considering HBase snapshots usage is in very early stage.

          Show
          Ted Yu added a comment - w.r.t. Lars' suggestion of introducing 'hbase.snapshot.use_newdir' config parameter, I think we'd better keep logic simple - considering HBase snapshots usage is in very early stage.
          Hide
          Ted Yu added a comment -

          can we add some logging code that checks the .snapshot dir for dirs with .snapshotinfo's in them

          If 2.0.5-alpha is the underlying hdfs, I think accessing .snapshot dir would not be allowed.

          Here was sample output from CLI:

          [hbase@ip-10-141-186-51 hbase]$ hadoop dfs -ls /apps/hbase/data/.snapshot
          ls: Cannot access /apps/hbase/data/.snapshot: No such file or directory.
          
          Show
          Ted Yu added a comment - can we add some logging code that checks the .snapshot dir for dirs with .snapshotinfo's in them If 2.0.5-alpha is the underlying hdfs, I think accessing .snapshot dir would not be allowed. Here was sample output from CLI: [hbase@ip-10-141-186-51 hbase]$ hadoop dfs -ls /apps/hbase/data/.snapshot ls: Cannot access /apps/hbase/data/.snapshot: No such file or directory.
          Hide
          Jonathan Hsieh added a comment -

          I like Matteo's latest version.

          Also, can we add some logging code that checks the .snapshot dir for dirs with .snapshotinfo's in them and warn with upgrade instructions in the logs? I think it would provide a sufficient breadcrumb for those who don't read release notes (or a link to the release notes)?

          Show
          Jonathan Hsieh added a comment - I like Matteo's latest version. Also, can we add some logging code that checks the .snapshot dir for dirs with .snapshotinfo's in them and warn with upgrade instructions in the logs? I think it would provide a sufficient breadcrumb for those who don't read release notes (or a link to the release notes)?
          Hide
          Jonathan Hsieh added a comment -

          You want doc and release notes? Or is either sufficient?

          Good release notes is sufficient. We'll get it done here.

          Is there any way to make rolling upgrades work? Even making the snapshot dir configurable would not allow for that.

          I was thinking we need a "version lock" or "upgrade lock" to block certain operations while rolling upgrades or rolling restarts for config changes are happening. Out of scope here though.

          Show
          Jonathan Hsieh added a comment - You want doc and release notes? Or is either sufficient? Good release notes is sufficient. We'll get it done here. Is there any way to make rolling upgrades work? Even making the snapshot dir configurable would not allow for that. I was thinking we need a "version lock" or "upgrade lock" to block certain operations while rolling upgrades or rolling restarts for config changes are happening. Out of scope here though.
          Hide
          Lars Hofhansl added a comment -

          Here's an option to make this configurable and still deal with HBASE_NON_TABLE_DIRS:

          • We'll add a config option: hbase.snapshot.use_newdir, which would default to false in 0.94.
          • If hbase.snapshot.use_newdir is true we use .hbase_snapshot, if false we use .snapshot
          • We'd add both .snapshot and .hbase_snapshot to HBASE_NON_TABLE_DIRS

          Thoughts? Too much effort?

          Show
          Lars Hofhansl added a comment - Here's an option to make this configurable and still deal with HBASE_NON_TABLE_DIRS: We'll add a config option: hbase.snapshot.use_newdir, which would default to false in 0.94. If hbase.snapshot.use_newdir is true we use .hbase_snapshot, if false we use .snapshot We'd add both .snapshot and .hbase_snapshot to HBASE_NON_TABLE_DIRS Thoughts? Too much effort?
          Hide
          Matteo Bertozzi added a comment -

          Should order of steps #2 and #3 above be switched ?

          yeah sort of I've missed a start the master step after the rename but maybe is better in this way:

          1. make sure all on-going snapshots activities have completed 
          2. shutdown All the masters
          3. rename ".snapshot" under HBase root directory as ".hbase-snapshot" 
          4. upgrade the masters to 0.94.7+ release 
          (You now can perform clone or restore, but not taking snapshots)
          5. upgrade the region servers to 0.94.7+ release 
          (You should not take snapshots until all the regions server are aligned to 0.94.7+ release)
          
          Show
          Matteo Bertozzi added a comment - Should order of steps #2 and #3 above be switched ? yeah sort of I've missed a start the master step after the rename but maybe is better in this way: 1. make sure all on-going snapshots activities have completed 2. shutdown All the masters 3. rename ".snapshot" under HBase root directory as ".hbase-snapshot" 4. upgrade the masters to 0.94.7+ release (You now can perform clone or restore, but not taking snapshots) 5. upgrade the region servers to 0.94.7+ release (You should not take snapshots until all the regions server are aligned to 0.94.7+ release)
          Hide
          Ted Yu added a comment -

          Should order of steps #2 and #3 above be switched ?

          Show
          Ted Yu added a comment - Should order of steps #2 and #3 above be switched ?
          Hide
          Matteo Bertozzi added a comment -

          forgot to say that if you take a snapshot during the RS upgrades nothing bad happen... you just end up with the .snapshot folder again, and the snapshot will fail since half of the data will be in .hbase-snapshot and the other half in .snapshot

          Show
          Matteo Bertozzi added a comment - forgot to say that if you take a snapshot during the RS upgrades nothing bad happen... you just end up with the .snapshot folder again, and the snapshot will fail since half of the data will be in .hbase-snapshot and the other half in .snapshot
          Hide
          Matteo Bertozzi added a comment -

          I think the upgrade procedure should be more like this:
          (otherwise you're implying a stop the whole cluster)

          1. make sure all on-going snapshots activities have completed 
          2. shutdown all the masters and upgrade them to 0.94.7+ release 
          3. rename ".snapshot" under HBase root directory as ".hbase-snapshot" 
          (You now can perform clone or restore, but not taking snapshots)
          4. upgrade the region servers
          (You should not take snapshots until all the regions server are aligned to 0.94.7+ release)
          
          Show
          Matteo Bertozzi added a comment - I think the upgrade procedure should be more like this: (otherwise you're implying a stop the whole cluster) 1. make sure all on-going snapshots activities have completed 2. shutdown all the masters and upgrade them to 0.94.7+ release 3. rename ".snapshot" under HBase root directory as ".hbase-snapshot" (You now can perform clone or restore, but not taking snapshots) 4. upgrade the region servers (You should not take snapshots until all the regions server are aligned to 0.94.7+ release)
          Hide
          Lars Hofhansl added a comment -

          Hmm... Jon just pointed out on the dev list that HDFS snapshots are slated for 2.0.5.

          Show
          Lars Hofhansl added a comment - Hmm... Jon just pointed out on the dev list that HDFS snapshots are slated for 2.0.5.
          Hide
          Lars Hofhansl added a comment -

          Can you draft something Jonathan Hsieh (otherwise we might go through unnecessary iterations)? You want doc and release notes? Or is either sufficient?

          Thinking now... Is there any way to make rolling upgrades work? Even making the snapshot dir configurable would not allow for that.
          A possible option would be to allow 0.94.7 to read snapshots from both directories, and only phase out the old directory in 0.94.8... But even then, one couldn't perform a rolling upgrade from 0.94.6 to 0.94.8. Besides that would be a lot of work.

          It's a shame that for 99% of all users this will never become an issue (as they won't deploy 0.94 on top of Hadoop 2.1+). From that angle I can see the usefulness of making this configurable (then we'd have to deal somehow with HBASE_NON_TABLE_DIRS).

          Blah, blah, blah ... Sorry, let's try to do a simple solution here.

          Show
          Lars Hofhansl added a comment - Can you draft something Jonathan Hsieh (otherwise we might go through unnecessary iterations)? You want doc and release notes? Or is either sufficient? Thinking now... Is there any way to make rolling upgrades work? Even making the snapshot dir configurable would not allow for that. A possible option would be to allow 0.94.7 to read snapshots from both directories, and only phase out the old directory in 0.94.8... But even then, one couldn't perform a rolling upgrade from 0.94.6 to 0.94.8. Besides that would be a lot of work. It's a shame that for 99% of all users this will never become an issue (as they won't deploy 0.94 on top of Hadoop 2.1+). From that angle I can see the usefulness of making this configurable (then we'd have to deal somehow with HBASE_NON_TABLE_DIRS). Blah, blah, blah ... Sorry, let's try to do a simple solution here.
          Hide
          Ted Yu added a comment -

          The test failure was not related to the patch:

          Running org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor
          2013-04-16 09:18:19.944 java[1050:1203] Unable to load realm info from SCDynamicStore
          Tests run: 62, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 151.559 sec

          Show
          Ted Yu added a comment - The test failure was not related to the patch: Running org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor 2013-04-16 09:18:19.944 java [1050:1203] Unable to load realm info from SCDynamicStore Tests run: 62, Failures: 0, Errors: 0, Skipped: 3, Time elapsed: 151.559 sec
          Hide
          Ted Yu added a comment -

          See if the updated release notes are better.

          Show
          Ted Yu added a comment - See if the updated release notes are better.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12578948/8352-trunk.txt
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 9 new or modified tests.

          +1 hadoop2.0. The patch compiles against the hadoop 2.0 profile.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 lineLengths. The patch does not introduce lines longer than 100

          +1 site. The mvn site goal succeeds with this patch.

          -1 core tests. The patch failed these unit tests:
          org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor

          Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html
          Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12578948/8352-trunk.txt against trunk revision . +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 9 new or modified tests. +1 hadoop2.0 . The patch compiles against the hadoop 2.0 profile. +1 javadoc . The javadoc tool did not generate any warning messages. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 findbugs . The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. +1 lineLengths . The patch does not introduce lines longer than 100 +1 site . The mvn site goal succeeds with this patch. -1 core tests . The patch failed these unit tests: org.apache.hadoop.hbase.client.TestFromClientSideWithCoprocessor Test results: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-protocol.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-client.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-examples.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop1-compat.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-prefix-tree.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-common.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-server.html Findbugs warnings: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//artifact/trunk/patchprocess/newPatchFindbugsWarningshbase-hadoop-compat.html Console output: https://builds.apache.org/job/PreCommit-HBASE-Build/5324//console This message is automatically generated.
          Hide
          Jonathan Hsieh added a comment -

          -1 until we get the docs / release notes updated with proper upgrading instructions (with mention of rolling upgrade caveat, how to upgrade). Alternately, the -1 goes away if a 0.94.7 blocker jira for docs with instructions for is filed.

          Show
          Jonathan Hsieh added a comment - -1 until we get the docs / release notes updated with proper upgrading instructions (with mention of rolling upgrade caveat, how to upgrade). Alternately, the -1 goes away if a 0.94.7 blocker jira for docs with instructions for is filed.
          Hide
          Matteo Bertozzi added a comment -

          The release note should highlight that you can't do rolling upgrade between 0.94.6 and 0.94.7 if you want to use snapshots, or better... if you want take a snapshot all the RSs and Masters must be aligned to 0.94.7+ version

          Show
          Matteo Bertozzi added a comment - The release note should highlight that you can't do rolling upgrade between 0.94.6 and 0.94.7 if you want to use snapshots, or better... if you want take a snapshot all the RSs and Masters must be aligned to 0.94.7+ version
          Hide
          Ted Yu added a comment -

          If we want to tackle those, I'd suggest a different a jira.

          +1

          Will attach trunk patch soon.

          Show
          Ted Yu added a comment - If we want to tackle those, I'd suggest a different a jira. +1 Will attach trunk patch soon.
          Hide
          Lars Hofhansl added a comment -

          +1 on this approach and patch.

          (Making this configurable would be overkill, and entail a slew of other changes; it would also raise questions about all the other hardcoded directory names such .logs .oldlogs .archive, etc.
          If we want to tackle those, I'd suggest a different a jira.)

          Show
          Lars Hofhansl added a comment - +1 on this approach and patch. (Making this configurable would be overkill, and entail a slew of other changes; it would also raise questions about all the other hardcoded directory names such .logs .oldlogs .archive, etc. If we want to tackle those, I'd suggest a different a jira.)
          Hide
          Ted Yu added a comment -

          Patch v1 for 0.94 branch.

          Test suite passed.

          Please provide your comments.

          Show
          Ted Yu added a comment - Patch v1 for 0.94 branch. Test suite passed. Please provide your comments.

            People

            • Assignee:
              Ted Yu
              Reporter:
              Ted Yu
            • Votes:
              0 Vote for this issue
              Watchers:
              12 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development