Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.0
    • Component/s: namenode
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      On trunk serialisation of editlog ops is in FSEditLog#log* and deserialisation is in FSEditLogOp.*Op . This improvement is to move the serialisation code into one place, i.e under FSEditLogOp.*Op.

      This is part of HDFS-1580.

      1. HDFS-2149.diff
        49 kB
        Ivan Kelly
      2. HDFS-2149.diff
        76 kB
        Ivan Kelly
      3. HDFS-2149.diff
        53 kB
        Ivan Kelly
      4. HDFS-2149.diff
        56 kB
        Ivan Kelly
      5. HDFS-2149.diff
        56 kB
        Ivan Kelly

        Issue Links

          Activity

          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-1073-branch #23 (See https://builds.apache.org/job/Hadoop-Hdfs-1073-branch/23/)
          Merge trunk into HDFS-1073.

          Resolved several conflicts due to merge of HDFS-2149 and HDFS-2212.
          Changes during resolution were:

          • move the writing of the transaction ID out of EditLogOutputStream to
            FSEditLogOp.Writer to match trunk's organization
          • remove JSPOOL related FsEditLogOp subclasses, add LogSegmentOp subclasses
          • modify TestEditLogJournalFailures to not keep trying to use streams after
            the simulated halt, since newer stricter assertions caused these writes to
            fail

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152128
          Files :

          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileAppend2.java
          • /hadoop/common/branches/HDFS-1073/hdfs/CHANGES.txt
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/blockmanagement/TestHeartbeatHandling.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/blockmanagement/BlockManagerTestUtil.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileStatus.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/docs/src/documentation/content/xdocs/hdfsproxy.xml
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestComputeInvalidateWork.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/contrib/build.xml
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeDeath.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DirectoryScanner.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreationDelete.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileAppend3.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/common/JspHelper.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/DFSClient.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileConcurrentReader.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/docs/src/documentation/content/xdocs/site.xml
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/unit/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDecommission.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestEditsDoubleBuffer.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestLeaseRecovery2.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/common/Storage.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/webapps/datanode
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestMultiThreadedHflush.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileAppend4.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/contrib/hdfsproxy
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreation.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/aop/org/apache/hadoop/hdfs/TestFiPipelines.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCorruption.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/blockmanagement/TestComputeInvalidateWork.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestPipelines.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogBackupOutputStream.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/webapps/hdfs
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestReadWhileWriting.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/webapps/secondary
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestRenameWhileOpen.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreationEmpty.java
          • /hadoop/common/branches/HDFS-1073/hdfs
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/c++/libhdfs
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStorageRestore.java
          • /hadoop/common/branches/HDFS-1073/hdfs/build.xml
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreationClient.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReplicationBlocks.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java
          • /hadoop/common/branches/HDFS-1073/hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-1073-branch #23 (See https://builds.apache.org/job/Hadoop-Hdfs-1073-branch/23/ ) Merge trunk into HDFS-1073 . Resolved several conflicts due to merge of HDFS-2149 and HDFS-2212 . Changes during resolution were: move the writing of the transaction ID out of EditLogOutputStream to FSEditLogOp.Writer to match trunk's organization remove JSPOOL related FsEditLogOp subclasses, add LogSegmentOp subclasses modify TestEditLogJournalFailures to not keep trying to use streams after the simulated halt, since newer stricter assertions caused these writes to fail todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1152128 Files : /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileAppend2.java /hadoop/common/branches/ HDFS-1073 /hdfs/CHANGES.txt /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSImage.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeManager.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DecommissionManager.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/blockmanagement/TestHeartbeatHandling.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/blockmanagement/BlockManagerTestUtil.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileStatus.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/docs/src/documentation/content/xdocs/hdfsproxy.xml /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestComputeInvalidateWork.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/contrib/build.xml /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestSaveNamespace.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDatanodeDeath.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DirectoryScanner.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreationDelete.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDFSUpgradeFromImage.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/DatanodeDescriptor.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NNStorage.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestClusterId.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileAppend3.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/common/JspHelper.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/DFSClient.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestEditLogJournalFailures.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileConcurrentReader.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/docs/src/documentation/content/xdocs/site.xml /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/unit/org/apache/hadoop/hdfs/server/datanode/TestBlockRecovery.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/balancer/TestBalancerWithMultipleNameNodes.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestDecommission.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestEditsDoubleBuffer.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestLeaseRecovery2.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/common/Storage.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/webapps/datanode /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestMultiThreadedHflush.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileAppend4.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/UpgradeUtilities.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/contrib/hdfsproxy /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/BlockPoolSliceStorage.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreation.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/aop/org/apache/hadoop/hdfs/TestFiPipelines.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCorruption.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicy.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/blockmanagement/TestComputeInvalidateWork.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestPipelines.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/SecondaryNameNode.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockManager.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogBackupOutputStream.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/webapps/hdfs /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/NameNode.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/BackupImage.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestReadWhileWriting.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/webapps/secondary /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/NameNodeAdapter.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestRenameWhileOpen.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataStorage.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreationEmpty.java /hadoop/common/branches/ HDFS-1073 /hdfs /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/BlockPlacementPolicyDefault.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/datanode/TestBlockReport.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/c++/libhdfs /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestStorageRestore.java /hadoop/common/branches/ HDFS-1073 /hdfs/build.xml /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/TestFileCreationClient.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/test/hdfs/org/apache/hadoop/hdfs/server/namenode/TestDeadDatanode.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/blockmanagement/PendingReplicationBlocks.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditsDoubleBuffer.java /hadoop/common/branches/ HDFS-1073 /hdfs/src/java/org/apache/hadoop/hdfs/server/datanode/DataNode.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #737 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/737/)
          HDFS-2149. Move EditLogOp serialization formats into FsEditLogOp implementations. Contributed by Ivan Kelly.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1151238
          Files :

          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java
          • /hadoop/common/trunk/hdfs/CHANGES.txt
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogBackupOutputStream.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #737 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/737/ ) HDFS-2149 . Move EditLogOp serialization formats into FsEditLogOp implementations. Contributed by Ivan Kelly. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1151238 Files : /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogBackupOutputStream.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #806 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/806/)
          HDFS-2149. Move EditLogOp serialization formats into FsEditLogOp implementations. Contributed by Ivan Kelly.

          todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1151238
          Files :

          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java
          • /hadoop/common/trunk/hdfs/CHANGES.txt
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogBackupOutputStream.java
          • /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #806 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/806/ ) HDFS-2149 . Move EditLogOp serialization formats into FsEditLogOp implementations. Contributed by Ivan Kelly. todd : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1151238 Files : /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogFileOutputStream.java /hadoop/common/trunk/hdfs/CHANGES.txt /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/FSEditLog.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogBackupOutputStream.java /hadoop/common/trunk/hdfs/src/java/org/apache/hadoop/hdfs/server/namenode/EditLogOutputStream.java
          Hide
          Todd Lipcon added a comment -

          Committed to trunk, thanks Ivan!

          Show
          Todd Lipcon added a comment - Committed to trunk, thanks Ivan!
          Hide
          Todd Lipcon added a comment -

          +1. Hudson QA bot timed out on the last test run, but seems unrelated to the patch. Since the two most recent patches only differ by the introduction of a 'static' keyword, I will commit based on the previous hudson +1 results. Thanks, Ivan!

          Show
          Todd Lipcon added a comment - +1. Hudson QA bot timed out on the last test run, but seems unrelated to the patch. Since the two most recent patches only differ by the introduction of a 'static' keyword, I will commit based on the previous hudson +1 results. Thanks, Ivan!
          Hide
          Ivan Kelly added a comment -

          Fixed findbugs error

          Show
          Ivan Kelly added a comment - Fixed findbugs error
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12487829/HDFS-2149.diff
          against trunk revision 1150960.

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed core unit tests.

          +1 contrib tests. The patch passed contrib unit tests.

          +1 system test framework. The patch passed system test framework compile.

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1018//testReport/
          Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1018//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1018//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12487829/HDFS-2149.diff against trunk revision 1150960. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. -1 findbugs. The patch appears to introduce 1 new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. +1 contrib tests. The patch passed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/1018//testReport/ Findbugs warnings: https://builds.apache.org/job/PreCommit-HDFS-Build/1018//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/1018//console This message is automatically generated.
          Hide
          Ivan Kelly added a comment -

          I've addressed all these comments. For writeFields I've created a new FSEditLogOp.Writer which is symmetrical to FSEditLogOp.Reader. There is also an asymmetry in how the checksumming is done. I have solved this in HDFS-2187 though.

          Show
          Ivan Kelly added a comment - I've addressed all these comments. For writeFields I've created a new FSEditLogOp.Writer which is symmetrical to FSEditLogOp.Reader. There is also an asymmetry in how the checksumming is done. I have solved this in HDFS-2187 though.
          Hide
          Todd Lipcon added a comment -
          • In all of the getInstance methods, you're doing FooOp.class.cast(...). Why not use the normal casting syntax (FooOp)...?
          • in logOpen and logCloseFile, you should use the subclass types rather than AddCloseOp. The fact that they both inherit from AddCloseOp should be an internal detail of FSEditLogOp
          • Indentation error:
                    EnumMap<FSEditLogOpCodes, FSEditLogOp> instances 
                    = new EnumMap<FSEditLogOpCodes, FSEditLogOp>(FSEditLogOpCodes.class);
            
          • I don't like the assymetry that Op.writeFields() writes the opcode, whereas readFields() doesn't (since it's read in readOp()). I think this could be fixed by pulling it up into the EditLogOutputStream write functions. Or, if you want it to all be in FsEditLogOp, add a new static function there called writeOp(FSEditLogOp, DataOutputStream) or something?
          Show
          Todd Lipcon added a comment - In all of the getInstance methods, you're doing FooOp.class.cast(...). Why not use the normal casting syntax (FooOp)...? in logOpen and logCloseFile, you should use the subclass types rather than AddCloseOp. The fact that they both inherit from AddCloseOp should be an internal detail of FSEditLogOp Indentation error: EnumMap<FSEditLogOpCodes, FSEditLogOp> instances = new EnumMap<FSEditLogOpCodes, FSEditLogOp>(FSEditLogOpCodes.class); I don't like the assymetry that Op.writeFields() writes the opcode, whereas readFields() doesn't (since it's read in readOp()). I think this could be fixed by pulling it up into the EditLogOutputStream write functions. Or, if you want it to all be in FsEditLogOp, add a new static function there called writeOp(FSEditLogOp, DataOutputStream) or something?
          Hide
          Todd Lipcon added a comment -

          marking patch available again to get hudson results

          Show
          Todd Lipcon added a comment - marking patch available again to get hudson results
          Hide
          Ivan Kelly added a comment -

          I've removed (2) for the current patch but I absolutely think that it's the way to go with input. To turn it around, is there ever any situation where you would want to read anything other than an FSEditLogOp from an edit log stream? The only situation previously was to read the version number at the start. This is metadata though. In BK I won't store this in the stream at all. In file and bk cases I think it's better if this is accessed with a getVersion() method rather than the reader having to know about the stored format.

          EditLogBackupOutputStream bug
          Well spotted. I've changed this again to use a byte[] to store the ops once it's written to the stream. It's now very similar to what it was originally.

          getInstance for AddCloseOp
          I've changed this so that AddCloseOp is abstract and you call AddOp.getInstance() and CloseOp.getInstance() to get the actual instances.

          javadoc on writeRaw
          That should have read "Should not be used on new code". In any case, i've just pulled the comment from 1073 instead.

          brevity changes
          I've removed most of the boilerplate to replace with a opInstances ThreadLocal. However, i've retained the getInstance() calls on the ops themselves. FSEditLogOp.getThreadLocalOp(OP_CLOSE, AddCloseOp.class) looks clunky to me. You're passing in the op code and the op class. You can't get away with just the op code because the generics framework need to know what to return. You can't get away with just the class because the enummap is code->instance, not visa versa. In any case, the getInstance methods are quite small. It was the ThreadLocal stuff that looked very heavy, IMO.

          Show
          Ivan Kelly added a comment - I've removed (2) for the current patch but I absolutely think that it's the way to go with input. To turn it around, is there ever any situation where you would want to read anything other than an FSEditLogOp from an edit log stream? The only situation previously was to read the version number at the start. This is metadata though. In BK I won't store this in the stream at all. In file and bk cases I think it's better if this is accessed with a getVersion() method rather than the reader having to know about the stored format. EditLogBackupOutputStream bug Well spotted. I've changed this again to use a byte[] to store the ops once it's written to the stream. It's now very similar to what it was originally. getInstance for AddCloseOp I've changed this so that AddCloseOp is abstract and you call AddOp.getInstance() and CloseOp.getInstance() to get the actual instances. javadoc on writeRaw That should have read "Should not be used on new code". In any case, i've just pulled the comment from 1073 instead. brevity changes I've removed most of the boilerplate to replace with a opInstances ThreadLocal. However, i've retained the getInstance() calls on the ops themselves. FSEditLogOp.getThreadLocalOp(OP_CLOSE, AddCloseOp.class) looks clunky to me. You're passing in the op code and the op class. You can't get away with just the op code because the generics framework need to know what to return. You can't get away with just the class because the enummap is code->instance, not visa versa. In any case, the getInstance methods are quite small. It was the ThreadLocal stuff that looked very heavy, IMO.
          Hide
          Todd Lipcon added a comment -

          Looking over the patch, it seems like this is really two distinct parts:

          1) cleans up the writing of edits:
          – rather than writing logEdit(opcode, Writable...) in FSEditLog, the API is logEdit(op record)
          – this API change filters down into EditLogOutputStream as well
          – writeRaw is introduced to be used from the BN (like in 1073 branch)
          – EditLogBackupOutputStream, which used to buffer Writables in JournalRecord, now directly buffers the op objects

          2) Changes the API around reading ops to be an iterator-style interface rather than a byte-style interface, by pushing down the FSEditLogOp.Reader into EditLogInputStream.

          I'm convinced that (1) above is a good idea, and I think the patch is mostly ready for commit modulo a couple things below.

          I'm not yet convinced that (2) above is quite right yet. We might need another iteration or two to discuss the APIs.

          Would you mind using this JIRA to get part (1) committed, and then work out the kinks in (2) next? Or is there some reason that part (1) depends on part (2) that I'm missing?

          Issues:

          EditLogBackupOutputStream bug:
          Buffering the Op objects in EditLogBackupOutputStream is likely to cause a bug since the objects are reused. If a single thread logs twice before syncing (unlikely but possible for some operations) the edits will get modified in place and the BN will diverge. This might be a bug in the current BN as well, but the Writables weren't always reused, whereas these op objects are.

          To fix this, I think we should make EditLogBackupOutputStream and EditLogFileOutputStream act the same - ie serialize the ops immediately into a buffer. Perhaps they could share this code in the abstract base class? Or, introduce a new classs like JournalDoubleBuffer?

          getInstance for AddCloseOp:
          Rather than have two separate ThreadLocals for AddCloseOp, I think it would be cleaner to add a "setOp()" function there on a single instance. My only reasoning is that then all of the classes are nicely symmetric.

          javadoc on writeRaw:
          I find this javadoc confusing. What do you mean by "should only be used in new code"? Also, the punctuation is incomplete which makes the comment hard to read. Perhaps just copy the javadoc from the 1073 branch s it doesn't need to be resolved at merge time?

          For brevity:
          Here's a thought: rather than giving each op its own ThreadLocal and getInstance method, why not make the opInstances EnumMap that we already have into a ThreadLocal? Something like:

          // in FSEditLogOp.java:
            ThreadLocal<EnumMap<FSEditLogOpCodes, FSEditLogOp>> opInstances =
              new ThreadLocal<EnumMap<FSEditLogOpCodes, FSEditLogOp>>() {
                @Override
                protected EnumMap<FSEditLogOpCodes, FSEditLogOp> initialValue() {
                  // ...
                }
            };
            
            <T extends FSEditLogOp> T getThreadLocalOp(
                FSEditLogOpCodes opCode, Class<T> clazz) {
              return clazz.cast(opInstances.get().get(opCode));
            }
          
          // in FSEditLog.java:
             public void logCloseFile(String path, INodeFile newNode) {
               AddCloseOp op = FSEditLogOp.getThreadLocalOp(OP_CLOSE, AddCloseOp.class)
                  .setPath(path)
                  ...;
             }
          

          I think this would reduce the amount of boilerplate and shouldn't be any discernible perf difference.

          Show
          Todd Lipcon added a comment - Looking over the patch, it seems like this is really two distinct parts: 1) cleans up the writing of edits: – rather than writing logEdit(opcode, Writable...) in FSEditLog, the API is logEdit(op record) – this API change filters down into EditLogOutputStream as well – writeRaw is introduced to be used from the BN (like in 1073 branch) – EditLogBackupOutputStream, which used to buffer Writables in JournalRecord, now directly buffers the op objects 2) Changes the API around reading ops to be an iterator-style interface rather than a byte-style interface, by pushing down the FSEditLogOp.Reader into EditLogInputStream. I'm convinced that (1) above is a good idea, and I think the patch is mostly ready for commit modulo a couple things below. I'm not yet convinced that (2) above is quite right yet. We might need another iteration or two to discuss the APIs. Would you mind using this JIRA to get part (1) committed, and then work out the kinks in (2) next? Or is there some reason that part (1) depends on part (2) that I'm missing? Issues: EditLogBackupOutputStream bug : Buffering the Op objects in EditLogBackupOutputStream is likely to cause a bug since the objects are reused. If a single thread logs twice before syncing (unlikely but possible for some operations) the edits will get modified in place and the BN will diverge. This might be a bug in the current BN as well, but the Writables weren't always reused, whereas these op objects are. To fix this, I think we should make EditLogBackupOutputStream and EditLogFileOutputStream act the same - ie serialize the ops immediately into a buffer. Perhaps they could share this code in the abstract base class? Or, introduce a new classs like JournalDoubleBuffer? getInstance for AddCloseOp : Rather than have two separate ThreadLocals for AddCloseOp, I think it would be cleaner to add a "setOp()" function there on a single instance. My only reasoning is that then all of the classes are nicely symmetric. javadoc on writeRaw : I find this javadoc confusing. What do you mean by "should only be used in new code"? Also, the punctuation is incomplete which makes the comment hard to read. Perhaps just copy the javadoc from the 1073 branch s it doesn't need to be resolved at merge time? For brevity : Here's a thought: rather than giving each op its own ThreadLocal and getInstance method, why not make the opInstances EnumMap that we already have into a ThreadLocal? Something like: // in FSEditLogOp.java: ThreadLocal<EnumMap<FSEditLogOpCodes, FSEditLogOp>> opInstances = new ThreadLocal<EnumMap<FSEditLogOpCodes, FSEditLogOp>>() { @Override protected EnumMap<FSEditLogOpCodes, FSEditLogOp> initialValue() { // ... } }; <T extends FSEditLogOp> T getThreadLocalOp( FSEditLogOpCodes opCode, Class <T> clazz) { return clazz. cast (opInstances.get().get(opCode)); } // in FSEditLog.java: public void logCloseFile( String path, INodeFile newNode) { AddCloseOp op = FSEditLogOp.getThreadLocalOp(OP_CLOSE, AddCloseOp.class) .setPath(path) ...; } I think this would reduce the amount of boilerplate and shouldn't be any discernible perf difference.
          Hide
          Todd Lipcon added a comment -

          Oh, I see. Let me review against trunk then, sorry for confusion.

          Show
          Todd Lipcon added a comment - Oh, I see. Let me review against trunk then, sorry for confusion.
          Hide
          Ivan Kelly added a comment -

          This is against trunk. There's nothing that would be specific to 1073 in it.

          Show
          Ivan Kelly added a comment - This is against trunk. There's nothing that would be specific to 1073 in it.
          Hide
          Todd Lipcon added a comment -

          hm, is this patch supposed to be against the tip of 1073 branch? Looks like it was made against a checkout prior to the BackupNode work

          Show
          Todd Lipcon added a comment - hm, is this patch supposed to be against the tip of 1073 branch? Looks like it was made against a checkout prior to the BackupNode work
          Hide
          Ivan Kelly added a comment -

          I ran these the NNThroughputBenchmark on these changes as follows:

          The bench was ran on a quad core xeon, with 48G of Ram. A dedicated disk was used for the namenode.

          Parameters: -op all -threads 100 -files 100000

          The bench was run 5 times both for trunk and HDFS-2149 and the average and standard deviation recorded as below. In short this patch actually gives a slight improvement in performance, but nothing significant.

          Numbers in the table are ops per second, bigger numbers are better.

          op Trunk Avg Trunk stddev HDFS-2149 Avg HDFS-2149 stddev Gain %Gain
          create 5174.918613 37.82819514 5180.145729 44.19491851 +5.227116624 +0.001010087
          open 29913.30775 2063.514703 30322.53589 1413.393394 +409.228147 +0.013680471
          delete 9135.00613 193.0711833 9329.309289 209.2509118 +194.3031585 +0.021270173
          fileStatus 134478.4952 32823.61628 140988.636 37665.42285 +6510.140784 +0.048410274
          rename 7199.803766 84.61922167 7084.922625 82.04274592 -114.8811417 -0.015956149
          blockReport 693.4065934 111.5533471 719.8717949 69.17435973 +26.46520147 +0.038166931
          replication 700 273.8612788 900 223.6067977 +200 +0.285714286
          clean 600 223.6067977 750 353.5533906 +150 +0.25
          Show
          Ivan Kelly added a comment - I ran these the NNThroughputBenchmark on these changes as follows: The bench was ran on a quad core xeon, with 48G of Ram. A dedicated disk was used for the namenode. Parameters: -op all -threads 100 -files 100000 The bench was run 5 times both for trunk and HDFS-2149 and the average and standard deviation recorded as below. In short this patch actually gives a slight improvement in performance, but nothing significant. Numbers in the table are ops per second, bigger numbers are better. op Trunk Avg Trunk stddev HDFS-2149 Avg HDFS-2149 stddev Gain %Gain create 5174.918613 37.82819514 5180.145729 44.19491851 +5.227116624 +0.001010087 open 29913.30775 2063.514703 30322.53589 1413.393394 +409.228147 +0.013680471 delete 9135.00613 193.0711833 9329.309289 209.2509118 +194.3031585 +0.021270173 fileStatus 134478.4952 32823.61628 140988.636 37665.42285 +6510.140784 +0.048410274 rename 7199.803766 84.61922167 7084.922625 82.04274592 -114.8811417 -0.015956149 blockReport 693.4065934 111.5533471 719.8717949 69.17435973 +26.46520147 +0.038166931 replication 700 273.8612788 900 223.6067977 +200 +0.285714286 clean 600 223.6067977 750 353.5533906 +150 +0.25
          Hide
          Ivan Kelly added a comment -

          Code ready for review/submission provides the benchmarks don't show anything nasty. I'm running those now.

          Show
          Ivan Kelly added a comment - Code ready for review/submission provides the benchmarks don't show anything nasty. I'm running those now.
          Hide
          Ivan Kelly added a comment -

          REPOST from HDFS-1073:

          HDFS-2149 will probably remove LogHeader completely. I plan to add a getVersion() call to InputStreams and each stream will handle it's own metadata internally. So EditLogFileInputStream will read it's version on creation, or first call to read etc. The input and output stream will be packet based, so an input stream is basically an iterator over FSEditLogOp objects and output stream is a sink for FSEditLogOp objects. I think the way I've implemented the FSEditLogOp objects should avoid all extra copies and object creation. Whats more, there's plenty to room to improve this by removing the creation of ArrayWritables and DeprecatedUTF8 objects and just write strings and arrays directly.

          Show
          Ivan Kelly added a comment - REPOST from HDFS-1073 : HDFS-2149 will probably remove LogHeader completely. I plan to add a getVersion() call to InputStreams and each stream will handle it's own metadata internally. So EditLogFileInputStream will read it's version on creation, or first call to read etc. The input and output stream will be packet based, so an input stream is basically an iterator over FSEditLogOp objects and output stream is a sink for FSEditLogOp objects. I think the way I've implemented the FSEditLogOp objects should avoid all extra copies and object creation. Whats more, there's plenty to room to improve this by removing the creation of ArrayWritables and DeprecatedUTF8 objects and just write strings and arrays directly.
          Hide
          Aaron T. Myers added a comment -

          I also find the OEV code confusing. I think its implementation of the visitor pattern is slightly different from what is usually done.

          As for a benchmark, I'd recommend either using NNThroughputBenchmark or writing your own custom micro-benchmark in the same vain as HDFS-1846.

          Show
          Aaron T. Myers added a comment - I also find the OEV code confusing. I think its implementation of the visitor pattern is slightly different from what is usually done. As for a benchmark, I'd recommend either using NNThroughputBenchmark or writing your own custom micro-benchmark in the same vain as HDFS-1846 .
          Hide
          Ivan Kelly added a comment -

          It's been mentioned in the past and it should be done. However that code confuses me at the moment. It should be possible to get rid of all the hardcoded duplicated op stuff though and replace it with reflection.

          I'd also like to run a benchmark on the new code to ensure it doesn't add any slowdown. Whats the best way to do this?

          Show
          Ivan Kelly added a comment - It's been mentioned in the past and it should be done. However that code confuses me at the moment. It should be possible to get rid of all the hardcoded duplicated op stuff though and replace it with reflection. I'd also like to run a benchmark on the new code to ensure it doesn't add any slowdown. Whats the best way to do this?
          Hide
          Aaron T. Myers added a comment -

          This is a great cleanup, Ivan. Have you considered also refactoring the OEV to use this code as well? Presently, all of the edit log loading code is effectively duplicated in the OEV.

          Show
          Aaron T. Myers added a comment - This is a great cleanup, Ivan. Have you considered also refactoring the OEV to use this code as well? Presently, all of the edit log loading code is effectively duplicated in the OEV.
          Hide
          Ivan Kelly added a comment -

          Initial drop of the changes. I have yet to figure out what to do with BackupNode which is using the editlog data streams as control streams also. I've commented out two lines related to this until I figure out what to do. Surprisingly, TestBackupNode passes without issue.

          FSEditLog is using EditLogOutputStreams as OutputStreams now, and I've added code to add checksumming for all streams. Previously it was only happening for File streams, not backup streams.

          Show
          Ivan Kelly added a comment - Initial drop of the changes. I have yet to figure out what to do with BackupNode which is using the editlog data streams as control streams also. I've commented out two lines related to this until I figure out what to do. Surprisingly, TestBackupNode passes without issue. FSEditLog is using EditLogOutputStreams as OutputStreams now, and I've added code to add checksumming for all streams. Previously it was only happening for File streams, not backup streams.

            People

            • Assignee:
              Ivan Kelly
              Reporter:
              Ivan Kelly
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development