Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: Snapshot (HDFS-2802)
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      When a regular file is reopened for append, the type is changed from INodeFile to INodeFileUnderConstruction. The type of snapshotted files (i.e. original files) is INodeFileWithLink. We have to support similar "under construction" INodeFileWithLink.

      1. h4098_20130117.patch
        39 kB
        Tsz Wo Nicholas Sze
      2. h4098_20130116.patch
        39 kB
        Tsz Wo Nicholas Sze
      3. HDFS-4098.compile.patch
        6 kB
        Jing Zhao
      4. h4098_20130113.patch
        31 kB
        Tsz Wo Nicholas Sze
      5. h4098_20130109b.patch
        31 kB
        Tsz Wo Nicholas Sze
      6. h4098_20130107.patch
        28 kB
        Tsz Wo Nicholas Sze

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-Snapshots-Branch-build #74 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/74/)
        HDFS-4098. Add FileWithSnapshot, INodeFileUnderConstructionWithSnapshot and INodeFileUnderConstructionSnapshot for supporting append to snapshotted files. (Revision 1434966)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1434966
        Files :

        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-2802.txt
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSnapshotPathINodes.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotReplication.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-Snapshots-Branch-build #74 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/74/ ) HDFS-4098 . Add FileWithSnapshot, INodeFileUnderConstructionWithSnapshot and INodeFileUnderConstructionSnapshot for supporting append to snapshotted files. (Revision 1434966) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1434966 Files : /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/CHANGES. HDFS-2802 .txt /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeDirectory.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSnapshotPathINodes.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshotReplication.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this.

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this.
        Hide
        Jing Zhao added a comment -

        +1 for the new patch.

        Show
        Jing Zhao added a comment - +1 for the new patch.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Thanks Jing for the review. You are right that we need to replace the inode in the circular linked list.

        h4098_20130117.patch

        Show
        Tsz Wo Nicholas Sze added a comment - Thanks Jing for the review. You are right that we need to replace the inode in the circular linked list. h4098_20130117.patch
        Hide
        Jing Zhao added a comment -

        The new patch looks good, and passed all the test on my machine. My only question is in INodeFileUnderConstructionWithSnapshot#toINodeFile(long), do we need to link the original under-construction inode with the new INodeFileWithSnapshot in the same circular linked list? Besides the question +1 for the patch.

        Show
        Jing Zhao added a comment - The new patch looks good, and passed all the test on my machine. My only question is in INodeFileUnderConstructionWithSnapshot#toINodeFile(long), do we need to link the original under-construction inode with the new INodeFileWithSnapshot in the same circular linked list? Besides the question +1 for the patch.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        h4098_20130116.patch: adds asINodeFile() so that the new interface does not use generic type as a workaround for the javac bug.

        Show
        Tsz Wo Nicholas Sze added a comment - h4098_20130116.patch: adds asINodeFile() so that the new interface does not use generic type as a workaround for the javac bug.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-Snapshots-Branch-build #71 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/71/)
        svn merge -c -1432788 for reverting HDFS-4098. Add FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot in order to support append to snapshotted files. (Revision 1433284)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433284
        Files :

        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-2802.txt
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSnapshotPathINodes.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-Snapshots-Branch-build #71 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/71/ ) svn merge -c -1432788 for reverting HDFS-4098 . Add FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot in order to support append to snapshotted files. (Revision 1433284) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1433284 Files : /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/CHANGES. HDFS-2802 .txt /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSnapshotPathINodes.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Due to the java compiler bugs Jing mentioned, I have reverted the commit.

        Show
        Tsz Wo Nicholas Sze added a comment - Due to the java compiler bugs Jing mentioned, I have reverted the commit.
        Hide
        Jing Zhao added a comment -

        The patch HDFS-4098.compile.patch can pass the compiling, but is not correct when specifying the types for FileLink.Util#getBlockReplication and FileLink.Util#collectSubtreeBlocksAndClear in INodeFileUnderConstructionWithLink and INodeFileWithLink.

        Show
        Jing Zhao added a comment - The patch HDFS-4098 .compile.patch can pass the compiling, but is not correct when specifying the types for FileLink.Util#getBlockReplication and FileLink.Util#collectSubtreeBlocksAndClear in INodeFileUnderConstructionWithLink and INodeFileWithLink.
        Hide
        Jing Zhao added a comment -

        Seems 4098 does not compile with javac in some JVM (e.g., in MAC with Java version 1.6.0_37). The error msg is attached below. It should be caused by a known bug with Java 1.6 according to these links: https://bugs.eclipse.org/bugs/show_bug.cgi?id=98379, http://stackoverflow.com/questions/8265296/ormlite-examples-for-android-wont-compile

        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java:[55,30] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds java.lang.Object,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java:[61,35] type parameters of <N>short cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java:[70,46] type parameters of <N>int cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[55,28] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[55,69] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[72,33] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[80,28] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[80,56] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[92,36] type parameters of <N>void cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[101,18] type parameters of <N>void cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[120,35] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java:[139,50] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java:[70,30] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds java.lang.Object,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java:[76,35] type parameters of <N>short cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java:[85,46] type parameters of <N>int cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink

        Show
        Jing Zhao added a comment - Seems 4098 does not compile with javac in some JVM (e.g., in MAC with Java version 1.6.0_37). The error msg is attached below. It should be caused by a known bug with Java 1.6 according to these links: https://bugs.eclipse.org/bugs/show_bug.cgi?id=98379 , http://stackoverflow.com/questions/8265296/ormlite-examples-for-android-wont-compile [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java: [55,30] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds java.lang.Object,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java: [61,35] type parameters of <N>short cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java: [70,46] type parameters of <N>int cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [55,28] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [55,69] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [72,33] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [80,28] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [80,56] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [92,36] type parameters of <N>void cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [101,18] type parameters of <N>void cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [120,35] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java: [139,50] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds N,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java: [70,30] type parameters of <N>N cannot be determined; no unique maximal instance exists for type variable N with upper bounds java.lang.Object,org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java: [76,35] type parameters of <N>short cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink [ERROR] /Users/jing/Documents/workspace/hadoop_2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java: [85,46] type parameters of <N>int cannot be determined; no unique maximal instance exists for type variable N with upper bounds org.apache.hadoop.hdfs.server.namenode.INodeFile,org.apache.hadoop.hdfs.server.namenode.snapshot.FileWithLink
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-Snapshots-Branch-build #70 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/70/)
        HDFS-4098. Add FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot in order to support append to snapshotted files. (Revision 1432788)

        Result = FAILURE
        szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1432788
        Files :

        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/CHANGES.HDFS-2802.txt
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionSnapshot.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSnapshotPathINodes.java
        • /hadoop/common/branches/HDFS-2802/hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-Snapshots-Branch-build #70 (See https://builds.apache.org/job/Hadoop-Hdfs-Snapshots-Branch-build/70/ ) HDFS-4098 . Add FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot in order to support append to snapshotted files. (Revision 1432788) Result = FAILURE szetszwo : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1432788 Files : /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/CHANGES. HDFS-2802 .txt /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogLoader.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INode.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFile.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/INodeFileUnderConstruction.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/FileWithLink.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionSnapshot.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileUnderConstructionWithLink.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/snapshot/INodeFileWithLink.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/TestSnapshotPathINodes.java /hadoop/common/branches/ HDFS-2802 /hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/namenode/snapshot/TestSnapshot.java
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I have committed this.

        Show
        Tsz Wo Nicholas Sze added a comment - I have committed this.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Here is a brief description of the patch:

        • It adds FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot.
        • For append, if it is an INodeFile, convert it to an INodeFileUnderConstruction as before. Otherwise, it is an INodeFileWithLink. Then convert it to an INodeFileUnderConstructionWithLink.
        • When an INodeFileUnderConstructionWithLink is snapshotted, the snapshot copy is then an INodeFileUnderConstructionSnapshot.
        Show
        Tsz Wo Nicholas Sze added a comment - Here is a brief description of the patch: It adds FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot. For append, if it is an INodeFile, convert it to an INodeFileUnderConstruction as before. Otherwise, it is an INodeFileWithLink. Then convert it to an INodeFileUnderConstructionWithLink. When an INodeFileUnderConstructionWithLink is snapshotted, the snapshot copy is then an INodeFileUnderConstructionSnapshot.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        h4098_20130113.patch: moves the type parameter from FileWithLink to the getNext, setNext and insert methods.

        Show
        Tsz Wo Nicholas Sze added a comment - h4098_20130113.patch: moves the type parameter from FileWithLink to the getNext, setNext and insert methods.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        1. Will add a comment.

        2. Oops, I forgot to update the javadoc of asFile(). I think we can get rid of it if we change the generic type.

        3. The util methods were moved from INodeFileWithLink but not new methods. These methods are already covered by some tests added earlier.

        Show
        Tsz Wo Nicholas Sze added a comment - 1. Will add a comment. 2. Oops, I forgot to update the javadoc of asFile(). I think we can get rid of it if we change the generic type. 3. The util methods were moved from INodeFileWithLink but not new methods. These methods are already covered by some tests added earlier.
        Hide
        Suresh Srinivas added a comment -

        I am +1 with the above comments addressed.

        Show
        Suresh Srinivas added a comment - I am +1 with the above comments addressed.
        Hide
        Suresh Srinivas added a comment -

        Comments:

        1. FSNamesystem.java
          • Please add a comment that for INode in read-only snapshot access time is not updated. Optionally move updating access time to a separate method with the description of when update is not made.
        2. FileWithLink.java
          • Based on the javadoc comments alone, I cannot figure out the difference between #asFile() and #getNext(). asFile could also move to util, since it need not be a method in the interface. Given implementations are returning this for asFile(), I am not sure I understand its functionality.
        3. FileWithLink#Util needs unit tests. If you want to do it in another jira, please create one.
        Show
        Suresh Srinivas added a comment - Comments: FSNamesystem.java Please add a comment that for INode in read-only snapshot access time is not updated. Optionally move updating access time to a separate method with the description of when update is not made. FileWithLink.java Based on the javadoc comments alone, I cannot figure out the difference between #asFile() and #getNext(). asFile could also move to util, since it need not be a method in the interface. Given implementations are returning this for asFile(), I am not sure I understand its functionality. FileWithLink#Util needs unit tests. If you want to do it in another jira, please create one.
        Hide
        Suresh Srinivas added a comment -

        Can you please add brief description of changes you are making in the patch, for the benefit of folks not going though the code?

        Show
        Suresh Srinivas added a comment - Can you please add brief description of changes you are making in the patch, for the benefit of folks not going though the code?
        Hide
        Tsz Wo Nicholas Sze added a comment -

        h4098_20130109b.patch: enables all snapshot append tests.

        Show
        Tsz Wo Nicholas Sze added a comment - h4098_20130109b.patch: enables all snapshot append tests.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        h4098_20130107.patch: adds FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot.

        Show
        Tsz Wo Nicholas Sze added a comment - h4098_20130107.patch: adds FileWithLink, INodeFileUnderConstructionWithLink and INodeFileUnderConstructionSnapshot.

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            7 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development