Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Snapshot (HDFS-2802)
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      After a long running stress test for snapshots, disallow snapshot throws IllegalStateException

      -bash-4.1$ hadoop dfs -rmr -skipTrash /user/foo/hdfs-snapshots
      rmr: cannot remove /user/foo/hdfs-snapshots: No such file or directory.

      -bash-4.1$ /usr/lib/hadoop/bin/hadoop dfs -mkdir /user/foo/hdfs-snapshots/test2

      -bash-4.1$ /usr/lib/hadoop/bin/hadoop dfsadmin -allowSnapshot /user/foo/hdfs-snapshots/test2
      Allowing snaphot on /user/foo/hdfs-snapshots/test2 succeeded

      -bash-4.1$ /usr/lib/hadoop/bin/hadoop dfsadmin -disallowSnapshot /user/foo/hdfs-snapshots/test2
      disallowSnapshot: java.io.IOException: java.lang.IllegalStateException: this is already an INodeDirectoryWithSnapshot, this=test2

      1. HDFS-4735.000.patch
        0.9 kB
        Jing Zhao
      2. HDFS-4735.001.patch
        12 kB
        Jing Zhao

        Activity

        Hide
        Ramya Sunil added a comment -

        Logs from the namenode:

        INFO org.apache.hadoop.ipc.Server: IPC Server handler 31 on 8020, call disallowSnapshot(/user/foo/hdfs-snapshots/test2) from .*: error: java.io.IOException: java.lang.IllegalStateException: this is already an INodeDirectoryWithSnapshot, this=test2
        java.io.IOException: java.lang.IllegalStateException: this is already an INodeDirectoryWithSnapshot, this=test2
                at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.replaceSelf4INodeDirectoryWithSnapshot(INodeDirectory.java:201)
                at org.apache.hadoop.hdfs.server.namenode.snapshot.INodeDirectorySnapshottable.replaceSelf(INodeDirectorySnapshottable.java:447)
                at org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.resetSnapshottable(SnapshotManager.java:108)
                at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.disallowSnapshot(FSNamesystem.java:6773)
                at org.apache.hadoop.hdfs.server.namenode.NameNode.disallowSnapshot(NameNode.java:1545)
                at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
                at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
                at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
                at java.lang.reflect.Method.invoke(Method.java:597)
                at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578)
                at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405)
                at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:396)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195)
                at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
        
        Show
        Ramya Sunil added a comment - Logs from the namenode: INFO org.apache.hadoop.ipc.Server: IPC Server handler 31 on 8020, call disallowSnapshot(/user/foo/hdfs-snapshots/test2) from .*: error: java.io.IOException: java.lang.IllegalStateException: this is already an INodeDirectoryWithSnapshot, this=test2 java.io.IOException: java.lang.IllegalStateException: this is already an INodeDirectoryWithSnapshot, this=test2 at org.apache.hadoop.hdfs.server.namenode.INodeDirectory.replaceSelf4INodeDirectoryWithSnapshot(INodeDirectory.java:201) at org.apache.hadoop.hdfs.server.namenode.snapshot.INodeDirectorySnapshottable.replaceSelf(INodeDirectorySnapshottable.java:447) at org.apache.hadoop.hdfs.server.namenode.snapshot.SnapshotManager.resetSnapshottable(SnapshotManager.java:108) at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.disallowSnapshot(FSNamesystem.java:6773) at org.apache.hadoop.hdfs.server.namenode.NameNode.disallowSnapshot(NameNode.java:1545) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25) at java.lang.reflect.Method.invoke(Method.java:597) at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:578) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1405) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1401) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1195) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1399)
        Hide
        Jing Zhao added a comment -

        When we have nested snapshottable directories and if we try to reset the snapshottable descendant back to an regular directory, we need to replace the snapshottable descendant with an INodeDirectoryWithSnapshot. In this case, we should not do the preconditions check in INodeDirectory#replaceSelf4INodeDirectoryWithSnapshot.

        Upload a simple fix first. Will add a unit test later.

        Show
        Jing Zhao added a comment - When we have nested snapshottable directories and if we try to reset the snapshottable descendant back to an regular directory, we need to replace the snapshottable descendant with an INodeDirectoryWithSnapshot. In this case, we should not do the preconditions check in INodeDirectory#replaceSelf4INodeDirectoryWithSnapshot. Upload a simple fix first. Will add a unit test later.
        Hide
        Jing Zhao added a comment -

        Add the unit test. Also add some code clean.

        Show
        Jing Zhao added a comment - Add the unit test. Also add some code clean.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        +1 patch looks good.

        Show
        Tsz Wo Nicholas Sze added a comment - +1 patch looks good.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this. Thanks, Jing!

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this. Thanks, Jing!
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-Snapshots-Branch-build #167 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/167/)
        HDFS-4735. DisallowSnapshot throws IllegalStateException for nested snapshottable directories. Contributed by Jing Zhao (Revision 1471214)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471214
        Files :

        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-2802.txt
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/Diff.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestDisallowModifyROSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestNestedSnapshots.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotBlocksMap.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotReplication.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-Snapshots-Branch-build #167 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/167/ ) HDFS-4735 . DisallowSnapshot throws IllegalStateException for nested snapshottable directories. Contributed by Jing Zhao (Revision 1471214) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1471214 Files : /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/CHANGES. HDFS-2802 .txt /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/util/Diff.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestFSImageWithSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestDisallowModifyROSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestNestedSnapshots.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotBlocksMap.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotReplication.java

          People

          • Assignee:
            Jing Zhao
            Reporter:
            Ramya Sunil
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development