Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9949

Add a test case to ensure that the DataNode does not regenerate its UUID when a storage directory is cleared

    Details

    • Type: Test
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: 2.6.0
    • Fix Version/s: 2.8.0, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None

      Description

      In the following scenario, in releases without HDFS-8211, the DN may regenerate its UUIDs unintentionally.

      0. Consider a DN with two disks /data1/dfs/dn,/data2/dfs/dn
      1. Stop DN
      2. Unmount the second disk, /data2/dfs/dn
      3. Create (in the scenario, this was an accident) /data2/dfs/dn on the root path
      4. Start DN
      5. DN now considers /data2/dfs/dn empty so formats it, but during the format it uses datanode.getDatanodeUuid() which is null until register() is called.
      6. As a result, after the directory loading, datanode.checkDatanodUuid() gets called with successful condition, and it causes a new generation of UUID which is written to all disks /data1/dfs/dn/current/VERSION and /data2/dfs/dn/current/VERSION.
      7. Stop DN (in the scenario, this was when the mistake of unmounted disk was realised)
      8. Mount the second disk back again /data2/dfs/dn, causing the VERSION file to be the original one again on it (mounting masks the root path that we last generated upon).
      9. DN fails to start up cause it finds mismatched UUID between the two disks, with an error similar to:

      WARN org.apache.hadoop.hdfs.server.common.Storage: org.apache.hadoop.hdfs.server.common.InconsistentFSStateException: Directory /data/2/dfs/dn is in an inconsistent state: Root /data/2/dfs/dn: DatanodeUuid=fe3a848f-beb8-4fcb-9581-c6fb1c701cc4, does not match 8ea9493c-7097-4ee3-96a3-0cc4dfc1d6ac from other StorageDirectory.

      The DN should not generate a new UUID if one of the storage disks already have the older one.

      HDFS-8211 unintentionally fixes this by changing the datanode.getDatanodeUuid() function to rely on the DataStorage representation of the UUID vs. the DatanodeID object which only gets available (non-null) after the registration.

      It'd still be good to add a direct test case to the above scenario that passes on trunk and branch-2, but fails on branch-2.7 and lower, so we can catch a regression around this in future.

      1. HDFS-9949.001.patch
        3 kB
        Harsh J
      2. HDFS-9949.000.patch
        3 kB
        Harsh J
      3. HDFS-9949.000.branch-2.7.not-for-commit.patch
        3 kB
        Harsh J

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #9473 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9473/)
          HDFS-9949. Add a test case to ensure that the DataNode does not (cmccabe: rev dc951e606f40bb779632a8a3e3a46aeccc4a446a)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeUUID.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #9473 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9473/ ) HDFS-9949 . Add a test case to ensure that the DataNode does not (cmccabe: rev dc951e606f40bb779632a8a3e3a46aeccc4a446a) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/server/datanode/TestDataNodeUUID.java
          Hide
          cmccabe Colin P. McCabe added a comment -

          Committed to 2.8, thanks!

          Show
          cmccabe Colin P. McCabe added a comment - Committed to 2.8, thanks!
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 11s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 7m 17s trunk passed
          +1 compile 0m 49s trunk passed with JDK v1.8.0_74
          +1 compile 0m 41s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 19s trunk passed
          +1 mvnsite 0m 51s trunk passed
          +1 mvneclipse 0m 12s trunk passed
          +1 findbugs 2m 0s trunk passed
          +1 javadoc 1m 19s trunk passed with JDK v1.8.0_74
          +1 javadoc 2m 1s trunk passed with JDK v1.7.0_95
          +1 mvninstall 0m 48s the patch passed
          +1 compile 0m 50s the patch passed with JDK v1.8.0_74
          +1 javac 0m 50s the patch passed
          +1 compile 0m 41s the patch passed with JDK v1.7.0_95
          +1 javac 0m 41s the patch passed
          +1 checkstyle 0m 18s the patch passed
          +1 mvnsite 0m 52s the patch passed
          +1 mvneclipse 0m 10s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 16s the patch passed
          +1 javadoc 1m 9s the patch passed with JDK v1.8.0_74
          +1 javadoc 1m 46s the patch passed with JDK v1.7.0_95
          -1 unit 67m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          -1 unit 59m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 21s Patch does not generate ASF License warnings.
          154m 45s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation
            hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits
            hadoop.hdfs.server.datanode.TestDataNodeMetrics
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793903/HDFS-9949.001.patch
          JIRA Issue HDFS-9949
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4dcd854b876c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 02a250d
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14848/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14848/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 11s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 7m 17s trunk passed +1 compile 0m 49s trunk passed with JDK v1.8.0_74 +1 compile 0m 41s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 19s trunk passed +1 mvnsite 0m 51s trunk passed +1 mvneclipse 0m 12s trunk passed +1 findbugs 2m 0s trunk passed +1 javadoc 1m 19s trunk passed with JDK v1.8.0_74 +1 javadoc 2m 1s trunk passed with JDK v1.7.0_95 +1 mvninstall 0m 48s the patch passed +1 compile 0m 50s the patch passed with JDK v1.8.0_74 +1 javac 0m 50s the patch passed +1 compile 0m 41s the patch passed with JDK v1.7.0_95 +1 javac 0m 41s the patch passed +1 checkstyle 0m 18s the patch passed +1 mvnsite 0m 52s the patch passed +1 mvneclipse 0m 10s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 16s the patch passed +1 javadoc 1m 9s the patch passed with JDK v1.8.0_74 +1 javadoc 1m 46s the patch passed with JDK v1.7.0_95 -1 unit 67m 58s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 59m 46s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 21s Patch does not generate ASF License warnings. 154m 45s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.blockmanagement.TestRBWBlockInvalidation   hadoop.hdfs.server.namenode.ha.TestFailureToReadEdits   hadoop.hdfs.server.datanode.TestDataNodeMetrics JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.namenode.ha.TestInitializeSharedEdits   hadoop.metrics2.sink.TestRollingFileSystemSinkWithSecureHdfs Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793903/HDFS-9949.001.patch JIRA Issue HDFS-9949 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4dcd854b876c 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 02a250d Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14848/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14848/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14848/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.
          Hide
          qwertymaniac Harsh J added a comment -

          Thank you for reviewing Colin P. McCabe! Agreed, I should've used a smaller sleep. Updated the trunk patch with the suggested 50ms. Test runs on my local machine in ~3s.

          Show
          qwertymaniac Harsh J added a comment - Thank you for reviewing Colin P. McCabe ! Agreed, I should've used a smaller sleep. Updated the trunk patch with the suggested 50ms . Test runs on my local machine in ~3s.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Thanks, Harsh J. Great find, and great root-cause analysis.

          106      while (!cluster.getDataNodes().get(0).isDatanodeFullyStarted()) {
          106	        Thread.sleep(500);
          107	      }
          

          Can we have a sleep for 50 ms here instead, just to take advantage of cases where the registration is quicker?

          +1 once that's addressed.

          Show
          cmccabe Colin P. McCabe added a comment - Thanks, Harsh J . Great find, and great root-cause analysis. 106 while (!cluster.getDataNodes().get(0).isDatanodeFullyStarted()) { 106 Thread .sleep(500); 107 } Can we have a sleep for 50 ms here instead, just to take advantage of cases where the registration is quicker? +1 once that's addressed.
          Hide
          qwertymaniac Harsh J added a comment -

          -1 unit *

          Patch only adds a test-case; the failing tests appear flaky instead of related to the added test for trunk and branch-2 here.

          The other patch (branch-2.7 named) is proof-of-test, but not intended for commit as HDFS-8211 is not in branch-2.7, which will cause the test to always fail.

          Show
          qwertymaniac Harsh J added a comment - -1 unit * Patch only adds a test-case; the failing tests appear flaky instead of related to the added test for trunk and branch-2 here. The other patch (branch-2.7 named) is proof-of-test, but not intended for commit as HDFS-8211 is not in branch-2.7, which will cause the test to always fail.
          Hide
          hadoopqa Hadoop QA added a comment -
          -1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 19s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 test4tests 0m 0s The patch appears to include 1 new or modified test files.
          +1 mvninstall 9m 39s trunk passed
          +1 compile 1m 18s trunk passed with JDK v1.8.0_74
          +1 compile 1m 4s trunk passed with JDK v1.7.0_95
          +1 checkstyle 0m 27s trunk passed
          +1 mvnsite 1m 20s trunk passed
          +1 mvneclipse 0m 18s trunk passed
          +1 findbugs 2m 44s trunk passed
          +1 javadoc 1m 45s trunk passed with JDK v1.8.0_74
          +1 javadoc 2m 39s trunk passed with JDK v1.7.0_95
          +1 mvninstall 1m 12s the patch passed
          +1 compile 1m 14s the patch passed with JDK v1.8.0_74
          +1 javac 1m 15s the patch passed
          +1 compile 1m 0s the patch passed with JDK v1.7.0_95
          +1 javac 1m 0s the patch passed
          +1 checkstyle 0m 23s the patch passed
          +1 mvnsite 1m 16s the patch passed
          +1 mvneclipse 0m 14s the patch passed
          +1 whitespace 0m 0s Patch has no whitespace issues.
          +1 findbugs 2m 55s the patch passed
          +1 javadoc 1m 41s the patch passed with JDK v1.8.0_74
          +1 javadoc 2m 48s the patch passed with JDK v1.7.0_95
          -1 unit 102m 56s hadoop-hdfs in the patch failed with JDK v1.8.0_74.
          -1 unit 89m 23s hadoop-hdfs in the patch failed with JDK v1.7.0_95.
          +1 asflicense 0m 30s Patch does not generate ASF License warnings.
          230m 7s



          Reason Tests
          JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock
            hadoop.hdfs.TestMissingBlocksAlert
            hadoop.hdfs.server.namenode.ha.TestEditLogTailer
            hadoop.hdfs.TestDataTransferKeepalive
            hadoop.hdfs.security.TestDelegationTokenForProxyUser
            hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure
            hadoop.hdfs.server.namenode.TestEditLog
            hadoop.hdfs.server.datanode.TestDirectoryScanner
          JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMXBean
            hadoop.hdfs.server.blockmanagement.TestBlockManagerSafeMode
            hadoop.hdfs.server.namenode.TestDecommissioningStatus
            hadoop.hdfs.server.namenode.TestFileTruncate
            hadoop.hdfs.TestEncryptionZones
            hadoop.hdfs.server.namenode.ha.TestRequestHedgingProxyProvider
            hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs
            hadoop.hdfs.server.datanode.TestDirectoryScanner



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:0ca8df7
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793172/HDFS-9949.000.patch
          JIRA Issue HDFS-9949
          Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle
          uname Linux 4fb01c304074 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / 658ee95
          Default Java 1.7.0_95
          Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95
          findbugs v3.0.0
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt
          unit https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt
          JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14803/testReport/
          modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14803/console
          Powered by Apache Yetus 0.2.0 http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 reexec 0m 19s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 test4tests 0m 0s The patch appears to include 1 new or modified test files. +1 mvninstall 9m 39s trunk passed +1 compile 1m 18s trunk passed with JDK v1.8.0_74 +1 compile 1m 4s trunk passed with JDK v1.7.0_95 +1 checkstyle 0m 27s trunk passed +1 mvnsite 1m 20s trunk passed +1 mvneclipse 0m 18s trunk passed +1 findbugs 2m 44s trunk passed +1 javadoc 1m 45s trunk passed with JDK v1.8.0_74 +1 javadoc 2m 39s trunk passed with JDK v1.7.0_95 +1 mvninstall 1m 12s the patch passed +1 compile 1m 14s the patch passed with JDK v1.8.0_74 +1 javac 1m 15s the patch passed +1 compile 1m 0s the patch passed with JDK v1.7.0_95 +1 javac 1m 0s the patch passed +1 checkstyle 0m 23s the patch passed +1 mvnsite 1m 16s the patch passed +1 mvneclipse 0m 14s the patch passed +1 whitespace 0m 0s Patch has no whitespace issues. +1 findbugs 2m 55s the patch passed +1 javadoc 1m 41s the patch passed with JDK v1.8.0_74 +1 javadoc 2m 48s the patch passed with JDK v1.7.0_95 -1 unit 102m 56s hadoop-hdfs in the patch failed with JDK v1.8.0_74. -1 unit 89m 23s hadoop-hdfs in the patch failed with JDK v1.7.0_95. +1 asflicense 0m 30s Patch does not generate ASF License warnings. 230m 7s Reason Tests JDK v1.8.0_74 Failed junit tests hadoop.hdfs.server.blockmanagement.TestPendingInvalidateBlock   hadoop.hdfs.TestMissingBlocksAlert   hadoop.hdfs.server.namenode.ha.TestEditLogTailer   hadoop.hdfs.TestDataTransferKeepalive   hadoop.hdfs.security.TestDelegationTokenForProxyUser   hadoop.hdfs.server.datanode.TestDataNodeVolumeFailure   hadoop.hdfs.server.namenode.TestEditLog   hadoop.hdfs.server.datanode.TestDirectoryScanner JDK v1.7.0_95 Failed junit tests hadoop.hdfs.server.namenode.TestNameNodeMXBean   hadoop.hdfs.server.blockmanagement.TestBlockManagerSafeMode   hadoop.hdfs.server.namenode.TestDecommissioningStatus   hadoop.hdfs.server.namenode.TestFileTruncate   hadoop.hdfs.TestEncryptionZones   hadoop.hdfs.server.namenode.ha.TestRequestHedgingProxyProvider   hadoop.metrics2.sink.TestRollingFileSystemSinkWithHdfs   hadoop.hdfs.server.datanode.TestDirectoryScanner Subsystem Report/Notes Docker Image:yetus/hadoop:0ca8df7 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12793172/HDFS-9949.000.patch JIRA Issue HDFS-9949 Optional Tests asflicense compile javac javadoc mvninstall mvnsite unit findbugs checkstyle uname Linux 4fb01c304074 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / 658ee95 Default Java 1.7.0_95 Multi-JDK versions /usr/lib/jvm/java-8-oracle:1.8.0_74 /usr/lib/jvm/java-7-openjdk-amd64:1.7.0_95 findbugs v3.0.0 unit https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt unit https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt unit test logs https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.8.0_74.txt https://builds.apache.org/job/PreCommit-HDFS-Build/14803/artifact/patchprocess/patch-unit-hadoop-hdfs-project_hadoop-hdfs-jdk1.7.0_95.txt JDK v1.7.0_95 Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/14803/testReport/ modules C: hadoop-hdfs-project/hadoop-hdfs U: hadoop-hdfs-project/hadoop-hdfs Console output https://builds.apache.org/job/PreCommit-HDFS-Build/14803/console Powered by Apache Yetus 0.2.0 http://yetus.apache.org This message was automatically generated.

            People

            • Assignee:
              qwertymaniac Harsh J
              Reporter:
              qwertymaniac Harsh J
            • Votes:
              0 Vote for this issue
              Watchers:
              8 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development