Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-10263

Provide symmetric entries in reversed snapshot diff report

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Open
    • Priority: Major
    • Resolution: Unresolved
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: namenode, snapshots
    • Labels:
      None

      Description

      Steps to reproduce:

      1. Take a snapshot s1 at:

      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/bar
      -rw-r--r--   1 yzhang supergroup       1024 2016-04-05 14:48 /target/bar/f1
      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/foo
      -rw-r--r--   1 yzhang supergroup       1024 2016-04-05 14:48 /target/foo/f1
      

      2. Make the following change:

        private int changeData7(Path dir) throws Exception {
          final Path foo = new Path(dir, "foo");
          final Path foo2 = new Path(dir, "foo2");
          final Path foo_f1 = new Path(foo, "f1");
          final Path foo2_f2 = new Path(foo2, "f2");
          final Path foo2_f1 = new Path(foo2, "f1");
          final Path foo_d1 = new Path(foo, "d1");
          final Path foo_d1_f3 = new Path(foo_d1, "f3");
      
          int numDeletedAndModified = 0;
          dfs.rename(foo, foo2);
      
          dfs.delete(foo2_f1, true);
          
          DFSTestUtil.createFile(dfs, foo_f1, BLOCK_SIZE, DATA_NUM, 0L);
          DFSTestUtil.appendFile(dfs, foo_f1, (int) BLOCK_SIZE);
          dfs.rename(foo_f1, foo2_f2);
          numDeletedAndModified += 1; // "M ./foo"
          DFSTestUtil.createFile(dfs, foo_d1_f3, BLOCK_SIZE, DATA_NUM, 0L);
          return numDeletedAndModified;
        }
      

      that results in

      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/bar
      -rw-r--r--   1 yzhang supergroup       1024 2016-04-05 14:48 /target/bar/f1
      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/foo
      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/foo/d1
      -rw-r--r--   1 yzhang supergroup       1024 2016-04-05 14:48 /target/foo/d1/f3
      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/foo2
      -rw-r--r--   1 yzhang supergroup       2048 2016-04-05 14:48 /target/foo2/f2
      

      3. take snapshot s2 here

      4. Do the following to revert the change done in step 2

       private int revertChangeData7(Path dir) throws Exception {
          final Path foo = new Path(dir, "foo");
          final Path foo2 = new Path(dir, "foo2");
          final Path foo_f1 = new Path(foo, "f1");
          final Path foo2_f2 = new Path(foo2, "f2");
          final Path foo2_f1 = new Path(foo2, "f1");
          final Path foo_d1 = new Path(foo, "d1");
          final Path foo_d1_f3 = new Path(foo_d1, "f3");
      
          int numDeletedAndModified = 0;
          
          dfs.delete(foo_d1, true);
      
          dfs.rename(foo2_f2, foo_f1);
          
          dfs.delete(foo, true);
          
          DFSTestUtil.createFile(dfs, foo2_f1, BLOCK_SIZE, DATA_NUM, 0L);
          DFSTestUtil.appendFile(dfs, foo2_f1, (int) BLOCK_SIZE);
      
          dfs.rename(foo2,  foo);
          
          return numDeletedAndModified;
        }
      

      that get the following results:

      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/bar
      -rw-r--r--   1 yzhang supergroup       1024 2016-04-05 14:48 /target/bar/f1
      drwxr-xr-x   - yzhang supergroup          0 2016-04-05 14:48 /target/foo
      -rw-r--r--   1 yzhang supergroup       2048 2016-04-05 14:48 /target/foo/f1
      

      4. Take snapshot s3 here.

      Below is the different snapshots

      s1-s2: Difference between snapshot s1 and snapshot s2 under directory /target:
      M	.
      +	./foo
      R	./foo -> ./foo2
      M	./foo
      +	./foo/f2
      -	./foo/f1
      
      s2-s1: Difference between snapshot s2 and snapshot s1 under directory /target:
      M	.
      -	./foo
      R	./foo2 -> ./foo
      M	./foo
      -	./foo/f2
      +	./foo/f1
      
      s2-s3: Difference between snapshot s2 and snapshot s3 under directory /target:
      M	.
      -	./foo
      R	./foo2 -> ./foo
      M	./foo2
      +	./foo2/f1
      -	./foo2/f2
      
      s3-s2: Difference between snapshot s3 and snapshot s2 under directory /target:
      M	.
      +	./foo
      R	./foo -> ./foo2
      M	./foo2
      -	./foo2/f1
      +	./foo2/f2
      

      The s2-s1 snapshot is supposed to be the same as s2-s3, because the change from s2 to s3 is an exact reversion of the change from s1 to s2. We can see that s1 and s3 have same file structures.

      However, the resulted shown above is not. I expect the following part

      M	./foo
      -	./foo/f2
      +	./foo/f1
      

      in s2-s1 diff should be

      M	./foo2
      +	./foo2/f1
      -	./foo2/f2
      

      (same as in s2-s3)

      instead.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                jingzhao Jing Zhao
                Reporter:
                yzhangal Yongjun Zhang
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated: