Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-9106

Transfer failure during pipeline recovery causes permanent write failures

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.8.0, 2.7.2, 3.0.0-alpha1
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      When a new node is added to a write pipeline during flush/sync, if the partial block transfer fails, the write will fail permanently without retrying or continuing with whatever is in the pipeline.

      The transfer often fails in busy clusters due to timeout. There is no per-packet ACK between client and datanode or between source and target datanodes. If the total transfer time exceeds the configured timeout + 10 seconds (2 * 5 seconds slack), it is considered failed. Naturally, the failure rate is higher with bigger block sizes.

      I propose following changes:

      • Transfer timeout needs to be different from per-packet timeout.
      • transfer should be retried if fails.
      1. HDFS-9106-poc.patch
        4 kB
        Kihwal Lee
      2. HDFS-9106.patch
        4 kB
        Kihwal Lee
      3. HDFS-9106.branch-2.7.patch
        4 kB
        Kihwal Lee

        Issue Links

          Activity

          Hide
          djp Junping Du added a comment -

          Move it out of 2.6.4 to 2.6.5 as this JIRA haven't being updated for a while.

          Show
          djp Junping Du added a comment - Move it out of 2.6.4 to 2.6.5 as this JIRA haven't being updated for a while.
          Hide
          djp Junping Du added a comment -

          Hi Yi Liu, Jing Zhao and Kihwal Lee, do we think this bug should be fixed in branch-2.6 also?

          Show
          djp Junping Du added a comment - Hi Yi Liu , Jing Zhao and Kihwal Lee , do we think this bug should be fixed in branch-2.6 also?
          Hide
          sjlee0 Sangjin Lee added a comment -

          Does this issue exist in 2.6.x? Should this be backported to branch-2.6?

          Show
          sjlee0 Sangjin Lee added a comment - Does this issue exist in 2.6.x? Should this be backported to branch-2.6?
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #430 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/430/)
          HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #430 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/430/ ) HDFS-9106 . Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2370 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2370/)
          HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2370 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2370/ ) HDFS-9106 . Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk #1192 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1192/)
          HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk #1192 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/1192/ ) HDFS-9106 . Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2397 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2397/)
          HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2397 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2397/ ) HDFS-9106 . Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #453 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/453/)
          HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #453 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/453/ ) HDFS-9106 . Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #459 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/459/)
          HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)

          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #459 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/459/ ) HDFS-9106 . Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731) hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          kihwal Kihwal Lee added a comment -

          For branch-2.7, the location of the file is different. Otherwise the same logic applies. Committing this to branch-2.7.

          Show
          kihwal Kihwal Lee added a comment - For branch-2.7, the location of the file is different. Otherwise the same logic applies. Committing this to branch-2.7.
          Hide
          kihwal Kihwal Lee added a comment -

          committed to branch-2 and trunk.

          Show
          kihwal Kihwal Lee added a comment - committed to branch-2 and trunk.
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-trunk-Commit #8532 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8532/)
          HDFS-9106. Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-trunk-Commit #8532 (See https://builds.apache.org/job/Hadoop-trunk-Commit/8532/ ) HDFS-9106 . Transfer failure during pipeline recovery causes permanent write failures. Contributed by Kihwal Lee. (kihwal: rev 4c9497cbf02ecc82532a4e79e18912d8e0eb4731) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DataStreamer.java
          Hide
          kihwal Kihwal Lee added a comment -

          Thanks, Yi Liu and Jing Zhao. I will commit this shortly.

          Show
          kihwal Kihwal Lee added a comment - Thanks, Yi Liu and Jing Zhao . I will commit this shortly.
          Hide
          jingzhao Jing Zhao added a comment -

          +1 for the latest patch. Thanks Kihwal Lee.

          Show
          jingzhao Jing Zhao added a comment - +1 for the latest patch. Thanks Kihwal Lee .
          Hide
          kihwal Kihwal Lee added a comment -

          Jing Zhao Do you have any additional comment?

          Show
          kihwal Kihwal Lee added a comment - Jing Zhao Do you have any additional comment?
          Hide
          hitliuyi Yi Liu added a comment -

          Looks good, +1, thanks Kihwal Lee.
          Let's wait for Jing to see whether he has further comments on this.

          Show
          hitliuyi Yi Liu added a comment - Looks good, +1, thanks Kihwal Lee . Let's wait for Jing to see whether he has further comments on this.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 21m 38s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 56s There were no new javac warning messages.
          +1 javadoc 10m 21s There were no new javadoc warning messages.
          +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 32s The applied patch generated 1 new checkstyle issues (total was 82, now 83).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 30s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 45s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 40s Pre-build of native portion
          -1 hdfs tests 68m 42s Tests failed in hadoop-hdfs.
              120m 5s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestParallelUnixDomainRead
          Timed out tests org.apache.hadoop.hdfs.web.TestWebHdfsTokens
            org.apache.hadoop.hdfs.TestClose



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12761963/HDFS-9106.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / f3e5bc6
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12635/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12635/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12635/testReport/
          Java 1.7.0_55
          uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12635/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 21m 38s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 56s There were no new javac warning messages. +1 javadoc 10m 21s There were no new javadoc warning messages. +1 release audit 0m 24s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 32s The applied patch generated 1 new checkstyle issues (total was 82, now 83). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 30s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 45s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 40s Pre-build of native portion -1 hdfs tests 68m 42s Tests failed in hadoop-hdfs.     120m 5s   Reason Tests Failed unit tests hadoop.hdfs.TestParallelUnixDomainRead Timed out tests org.apache.hadoop.hdfs.web.TestWebHdfsTokens   org.apache.hadoop.hdfs.TestClose Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761963/HDFS-9106.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / f3e5bc6 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12635/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12635/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12635/testReport/ Java 1.7.0_55 uname Linux asf904.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12635/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -
          • Removed the try block around the RPC calls.
          • The original nodes with their storage types and ids are saved and restored if the new node fails.
          Show
          kihwal Kihwal Lee added a comment - Removed the try block around the RPC calls. The original nodes with their storage types and ids are saved and restored if the new node fails.
          Hide
          jingzhao Jing Zhao added a comment -

          Transfer timeout needs to be different from per-packet timeout.

          +1 for changing the timeout.

          if the partial block transfer fails, the write will fail permanently without retrying or continuing with whatever is in the pipeline

          If the partial block transfer fails, and if bestEffort is enabled, the current code will still use the remaining datanodes to setup the pipeline? But looks like the nodes may still include the new DN after the failure though.

          Show
          jingzhao Jing Zhao added a comment - Transfer timeout needs to be different from per-packet timeout. +1 for changing the timeout. if the partial block transfer fails, the write will fail permanently without retrying or continuing with whatever is in the pipeline If the partial block transfer fails, and if bestEffort is enabled, the current code will still use the remaining datanodes to setup the pipeline? But looks like the nodes may still include the new DN after the failure though.
          Hide
          hitliuyi Yi Liu added a comment -

          Thanks Kihwal Lee for working on this. I have few comments:

          1.

          +      try {
          +        //get a new datanode
          +        lb = dfsClient.namenode.getAdditionalDatanode(
          +            src, stat.getFileId(), block, nodes, storageIDs,
          +            exclude.toArray(new DatanodeInfo[exclude.size()]),
          +            1, dfsClient.clientName);
          +      } catch (IOException ioe) {
          +        DFSClient.LOG.warn("Error while asking for a new node to namenode: "
          +            + ioe.getMessage());
          +        caughtException = ioe;
          +        tried++;
          +        continue;
          +      }
          

          I see you catch the IOException of rpc to NameNode, for dfsClient.namenode, we already have retry policy for rpc to namenode. I wonder what IOExceptions do you want to handle here?

          2.
          Followings look reasonable.

          Transfer timeout needs to be different from per-packet timeout.
          transfer should be retried if fails.

          In the patch, it allows 3 tries, so ideally we can try 3 different datanodes. My doubt is originally why we have bestEffort instead of implementing the retries? Is it for performance consideration? It rarely happens after we retry 3 times then still can't find a good datanode to replace.

          Show
          hitliuyi Yi Liu added a comment - Thanks Kihwal Lee for working on this. I have few comments: 1. + try { + //get a new datanode + lb = dfsClient.namenode.getAdditionalDatanode( + src, stat.getFileId(), block, nodes, storageIDs, + exclude.toArray( new DatanodeInfo[exclude.size()]), + 1, dfsClient.clientName); + } catch (IOException ioe) { + DFSClient.LOG.warn( "Error while asking for a new node to namenode: " + + ioe.getMessage()); + caughtException = ioe; + tried++; + continue ; + } I see you catch the IOException of rpc to NameNode, for dfsClient.namenode , we already have retry policy for rpc to namenode. I wonder what IOExceptions do you want to handle here? 2. Followings look reasonable. Transfer timeout needs to be different from per-packet timeout. transfer should be retried if fails. In the patch, it allows 3 tries, so ideally we can try 3 different datanodes. My doubt is originally why we have bestEffort instead of implementing the retries? Is it for performance consideration? It rarely happens after we retry 3 times then still can't find a good datanode to replace.
          Hide
          hadoopqa Hadoop QA added a comment -



          -1 overall



          Vote Subsystem Runtime Comment
          0 pre-patch 18m 4s Pre-patch trunk compilation is healthy.
          +1 @author 0m 0s The patch does not contain any @author tags.
          -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch.
          +1 javac 8m 4s There were no new javac warning messages.
          +1 javadoc 10m 21s There were no new javadoc warning messages.
          +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings.
          -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 82, now 83).
          +1 whitespace 0m 0s The patch has no lines that end in whitespace.
          +1 install 1m 31s mvn install still works.
          +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse.
          +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings.
          +1 native 3m 17s Pre-build of native portion
          -1 hdfs tests 192m 12s Tests failed in hadoop-hdfs.
              238m 24s  



          Reason Tests
          Failed unit tests hadoop.hdfs.TestReplaceDatanodeOnFailure
            hadoop.hdfs.server.blockmanagement.TestNodeCount



          Subsystem Report/Notes
          Patch URL http://issues.apache.org/jira/secure/attachment/12761154/HDFS-9106-poc.patch
          Optional Tests javadoc javac unit findbugs checkstyle
          git revision trunk / 88d89267
          checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12544/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt
          hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12544/artifact/patchprocess/testrun_hadoop-hdfs.txt
          Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12544/testReport/
          Java 1.7.0_55
          uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12544/console

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall Vote Subsystem Runtime Comment 0 pre-patch 18m 4s Pre-patch trunk compilation is healthy. +1 @author 0m 0s The patch does not contain any @author tags. -1 tests included 0m 0s The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac 8m 4s There were no new javac warning messages. +1 javadoc 10m 21s There were no new javadoc warning messages. +1 release audit 0m 23s The applied patch does not increase the total number of release audit warnings. -1 checkstyle 1m 23s The applied patch generated 1 new checkstyle issues (total was 82, now 83). +1 whitespace 0m 0s The patch has no lines that end in whitespace. +1 install 1m 31s mvn install still works. +1 eclipse:eclipse 0m 34s The patch built with eclipse:eclipse. +1 findbugs 2m 31s The patch does not introduce any new Findbugs (version 3.0.0) warnings. +1 native 3m 17s Pre-build of native portion -1 hdfs tests 192m 12s Tests failed in hadoop-hdfs.     238m 24s   Reason Tests Failed unit tests hadoop.hdfs.TestReplaceDatanodeOnFailure   hadoop.hdfs.server.blockmanagement.TestNodeCount Subsystem Report/Notes Patch URL http://issues.apache.org/jira/secure/attachment/12761154/HDFS-9106-poc.patch Optional Tests javadoc javac unit findbugs checkstyle git revision trunk / 88d89267 checkstyle https://builds.apache.org/job/PreCommit-HDFS-Build/12544/artifact/patchprocess/diffcheckstylehadoop-hdfs.txt hadoop-hdfs test log https://builds.apache.org/job/PreCommit-HDFS-Build/12544/artifact/patchprocess/testrun_hadoop-hdfs.txt Test Results https://builds.apache.org/job/PreCommit-HDFS-Build/12544/testReport/ Java 1.7.0_55 uname Linux asf909.gq1.ygridcore.net 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Console output https://builds.apache.org/job/PreCommit-HDFS-Build/12544/console This message was automatically generated.
          Hide
          kihwal Kihwal Lee added a comment -

          Attaching a patch with no test to run it through jenkins.

          Show
          kihwal Kihwal Lee added a comment - Attaching a patch with no test to run it through jenkins.

            People

            • Assignee:
              kihwal Kihwal Lee
              Reporter:
              kihwal Kihwal Lee
            • Votes:
              0 Vote for this issue
              Watchers:
              16 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development