Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-8070

Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode

    Details

    • Target Version/s:

      Description

      HDFS ShortCircuitShm layer keeps the task locked up during multi-threaded split-generation.

      I hit this immediately after I upgraded the data, so I wonder if the ShortCircuitShim wire protocol has trouble when 2.8.0 DN talks to a 2.7.0 Client?

      2015-04-06 00:04:30,780 INFO [ORC_GET_SPLITS #3] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL ss_sold_date_sk)
      expr = (not leaf-0)
      2015-04-06 00:04:30,781 ERROR [ShortCircuitCache_SlotReleaser] shortcircuit.ShortCircuitCache: ShortCircuitCache(0x29e82045): failed to release short-circuit shared memory slot Slot(slotIdx=2, shm=DfsClientShm(a86ee34576d93c4964005d90b0d97c38)) by sending ReleaseShortCircuitAccessRequestProto to /grid/0/cluster/hdfs/dn_socket.  Closing shared memory segment.
      java.io.IOException: ERROR_INVALID: there is no shared memory segment registered with shmId a86ee34576d93c4964005d90b0d97c38
      	at org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache$SlotReleaser.run(ShortCircuitCache.java:208)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      2015-04-06 00:04:30,781 INFO [ORC_GET_SPLITS #5] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL ss_sold_date_sk)
      expr = (not leaf-0)
      2015-04-06 00:04:30,781 WARN [ShortCircuitCache_SlotReleaser] shortcircuit.DfsClientShmManager: EndpointShmManager(172.19.128.60:50010, parent=ShortCircuitShmManager(5e763476)): error shutting down shm: got IOException calling shutdown(SHUT_RDWR)
      java.nio.channels.ClosedChannelException
      	at org.apache.hadoop.util.CloseableReferenceCount.reference(CloseableReferenceCount.java:57)
      	at org.apache.hadoop.net.unix.DomainSocket.shutdown(DomainSocket.java:387)
      	at org.apache.hadoop.hdfs.shortcircuit.DfsClientShmManager$EndpointShmManager.shutdown(DfsClientShmManager.java:378)
      	at org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache$SlotReleaser.run(ShortCircuitCache.java:223)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      2015-04-06 00:04:30,783 INFO [ORC_GET_SPLITS #7] orc.OrcInputFormat: ORC pushdown predicate: leaf-0 = (IS_NULL cs_sold_date_sk)
      expr = (not leaf-0)
      2015-04-06 00:04:30,785 ERROR [ShortCircuitCache_SlotReleaser] shortcircuit.ShortCircuitCache: ShortCircuitCache(0x29e82045): failed to release short-circuit shared memory slot Slot(slotIdx=4, shm=DfsClientShm(a86ee34576d93c4964005d90b0d97c38)) by sending ReleaseShortCircuitAccessRequestProto to /grid/0/cluster/hdfs/dn_socket.  Closing shared memory segment.
      java.io.IOException: ERROR_INVALID: there is no shared memory segment registered with shmId a86ee34576d93c4964005d90b0d97c38
      	at org.apache.hadoop.hdfs.shortcircuit.ShortCircuitCache$SlotReleaser.run(ShortCircuitCache.java:208)
      	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
      	at java.util.concurrent.FutureTask.run(FutureTask.java:266)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
      	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
      	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
      	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
      	at java.lang.Thread.run(Thread.java:745)
      

      Looks like a double free-fd condition?

      2015-04-02 18:58:47,653 [DataXceiver for client unix:/grid/0/cluster/hdfs/dn_socket [Passing file descriptors for block BP-942051088-172.18.0.41-1370508013893:blk_1076973408_1099515627985]] INFO org.apache.hadoop.hdfs.server.datanode.DataNode: Unregistering SlotId(3bd7fd9aed791e95acfb5034e6617d83:0) because the requestShortCircuitFdsForRead operation failed.
      2015-04-02 18:58:47,653 [DataXceiver for client unix:/grid/0/cluster/hdfs/dn_socket [Passing file descriptors for block BP-942051088-<ip>-1370508013893:blk_1076973408_1099515627985]] INFO org.apache.hadoop.hdfs.server.datanode.DataNode.clienttrace: src: 127.0.0.1, dest: 127.0.0.1, op: REQUEST_SHORT_CIRCUIT_FDS, blockid: 1076973408, srvID: ba7b6f19-47e0-4b86-af50-23981649318c, success: false
      2015-04-02 18:58:47,654 [DataXceiver for client unix:/grid/0/cluster/hdfs/dn_socket [Passing file descriptors for block BP-942051088-172.18.0.41-1370508013893:blk_1076973408_1099515627985]] ERROR org.apache.hadoop.hdfs.server.datanode.DataNode: cn060-10.l42scl.hortonworks.com:50010:DataXceiver error processing REQUEST_SHORT_CIRCUIT_FDS operation  src: unix:/grid/0/cluster/hdfs/dn_socket dst: <local>
      java.io.EOFException
              at org.apache.hadoop.hdfs.server.datanode.DataXceiver.requestShortCircuitFds(DataXceiver.java:352)
              at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.opRequestShortCircuitFds(Receiver.java:187)
              at org.apache.hadoop.hdfs.protocol.datatransfer.Receiver.processOp(Receiver.java:89)
              at org.apache.hadoop.hdfs.server.datanode.DataXceiver.run(DataXceiver.java:251)
              at java.lang.Thread.run(Thread.java:745)
      

      Investigating more, since the exact exception from the DataNode call is not logged.

      1. HDFS-8070.001.patch
        5 kB
        Colin P. McCabe
      2. HDFS-8070.branch-2.6.patch
        5 kB
        Akira Ajisaka

        Issue Links

          Activity

          Hide
          cmccabe Colin P. McCabe added a comment -

          Both Hadoop 2.7 and the Haodop 2 branch (which is what I assume you mean by Hadoop 2.8) have HDFS-7915. So I think there should not be any compatibility issues on that front.

          Can you check whether the patch up at HADOOP-11802 solves your issue? At very least, it should get you a more informative exception.

          Show
          cmccabe Colin P. McCabe added a comment - Both Hadoop 2.7 and the Haodop 2 branch (which is what I assume you mean by Hadoop 2.8) have HDFS-7915 . So I think there should not be any compatibility issues on that front. Can you check whether the patch up at HADOOP-11802 solves your issue? At very least, it should get you a more informative exception.
          Hide
          hadoopqa Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12726024/HDFS-8070.001.patch
          against trunk revision 4308910.

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. There were no new javadoc warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 2.0.3) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs:

          org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery

          Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10290//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10290//console

          This message is automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - -1 overall . Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12726024/HDFS-8070.001.patch against trunk revision 4308910. +1 @author . The patch does not contain any @author tags. +1 tests included . The patch appears to include 1 new or modified test files. +1 javac . The applied patch does not increase the total number of javac compiler warnings. +1 javadoc . There were no new javadoc warning messages. +1 eclipse:eclipse . The patch built with eclipse:eclipse. +1 findbugs . The patch does not introduce any new Findbugs (version 2.0.3) warnings. +1 release audit . The applied patch does not increase the total number of release audit warnings. -1 core tests . The patch failed these unit tests in hadoop-hdfs-project/hadoop-hdfs: org.apache.hadoop.hdfs.server.datanode.fsdataset.impl.TestLazyPersistReplicaRecovery Test results: https://builds.apache.org/job/PreCommit-HDFS-Build/10290//testReport/ Console output: https://builds.apache.org/job/PreCommit-HDFS-Build/10290//console This message is automatically generated.
          Hide
          gopalv Gopal V added a comment -

          Colin P. McCabe: HADOOP-11802 seemed to be a client-side fix.

          For this patch, do I have to redeploy the HDFS DN to test?

          Show
          gopalv Gopal V added a comment - Colin P. McCabe : HADOOP-11802 seemed to be a client-side fix. For this patch, do I have to redeploy the HDFS DN to test?
          Hide
          cmccabe Colin P. McCabe added a comment -

          For this patch, do I have to redeploy the HDFS DN to test?

          Yes, this is a datanode-side fix.

          Show
          cmccabe Colin P. McCabe added a comment - For this patch, do I have to redeploy the HDFS DN to test? Yes, this is a datanode-side fix.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Marking it for 2.7.1 given the discussion on dev lists.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Marking it for 2.7.1 given the discussion on dev lists.
          Hide
          arpitagarwal Arpit Agarwal added a comment -

          +1 for the patch.

          Show
          arpitagarwal Arpit Agarwal added a comment - +1 for the patch.
          Hide
          cmccabe Colin P. McCabe added a comment -

          Committed to 2.7.1. Thanks for the reviews.

          Show
          cmccabe Colin P. McCabe added a comment - Committed to 2.7.1. Thanks for the reviews.
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #7659 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7659/)
          HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #7659 (See https://builds.apache.org/job/Hadoop-trunk-Commit/7659/ ) HDFS-8070 . Pre- HDFS-7915 DFSClient cannot use short circuit on post- HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/)
          HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk #2105 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/2105/ ) HDFS-8070 . Pre- HDFS-7915 DFSClient cannot use short circuit on post- HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/)
          HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0)

          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Hdfs-trunk-Java8 #164 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Java8/164/ ) HDFS-8070 . Pre- HDFS-7915 DFSClient cannot use short circuit on post- HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0) hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/)
          HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Yarn-trunk-Java8 #173 (See https://builds.apache.org/job/Hadoop-Yarn-trunk-Java8/173/ ) HDFS-8070 . Pre- HDFS-7915 DFSClient cannot use short circuit on post- HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/)
          HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0)

          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-Yarn-trunk #907 (See https://builds.apache.org/job/Hadoop-Yarn-trunk/907/ ) HDFS-8070 . Pre- HDFS-7915 DFSClient cannot use short circuit on post- HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0) hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/)
          HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk-Java8 #174 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Java8/174/ ) HDFS-8070 . Pre- HDFS-7915 DFSClient cannot use short circuit on post- HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          Hide
          hudson Hudson added a comment -

          FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/)
          HDFS-8070. Pre-HDFS-7915 DFSClient cannot use short circuit on post-HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0)

          • hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java
          • hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java
          • hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          Show
          hudson Hudson added a comment - FAILURE: Integrated in Hadoop-Mapreduce-trunk #2123 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/2123/ ) HDFS-8070 . Pre- HDFS-7915 DFSClient cannot use short circuit on post- HDFS-7915 DataNode (cmccabe) (cmccabe: rev a8898445dc9b5cdb7230e2e23a57393c9f378ff0) hadoop-hdfs-project/hadoop-hdfs/CHANGES.txt hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/BlockReaderFactory.java hadoop-hdfs-project/hadoop-hdfs/src/test/java/org/apache/hadoop/hdfs/shortcircuit/TestShortCircuitCache.java hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/protocol/datatransfer/Receiver.java
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Attaching a patch for branch-2.6. If we are going to include HADOOP-11802, we need to include HDFS-7915 and this issue as well.

          Show
          ajisakaa Akira Ajisaka added a comment - Attaching a patch for branch-2.6. If we are going to include HADOOP-11802 , we need to include HDFS-7915 and this issue as well.
          Hide
          vinodkv Vinod Kumar Vavilapalli added a comment -

          Thanks Akira Ajisaka. Pulled this into 2.6.1 after running compilation and TestShortCircuitCache.

          Show
          vinodkv Vinod Kumar Vavilapalli added a comment - Thanks Akira Ajisaka . Pulled this into 2.6.1 after running compilation and TestShortCircuitCache.

            People

            • Assignee:
              cmccabe Colin P. McCabe
              Reporter:
              gopalv Gopal V
            • Votes:
              1 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development