Details

    • Type: Sub-task Sub-task
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 1.2.0, 2.0.2-alpha
    • Component/s: native, performance
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      In HADOOP-7714, we determined that using fadvise inside of the MapReduce shuffle can yield very good performance improvements. But many parts of the shuffle are FileSystem-agnostic and thus operate on FSDataInputStreams and RawLocalFileSystems. This JIRA is to figure out how to allow RawLocalFileSystem to expose its FileDescriptor object without unnecessarily polluting the public APIs.

      1. HADOOP-7754_trunk_rev4.patch
        7 kB
        Ahmed Radwan
      2. test-patch-hadoop-7754.txt
        168 kB
        Ahmed Radwan
      3. HADOOP-7754_trunk_rev4.patch
        7 kB
        Ahmed Radwan
      4. HADOOP-7754_trunk_rev4.patch
        7 kB
        Ahmed Radwan
      5. HADOOP-7754_branch-1_rev4.patch
        6 kB
        Ahmed Radwan
      6. HADOOP-7754_trunk_rev3.patch
        6 kB
        Ahmed Radwan
      7. HADOOP-7754_branch-1_rev3.patch
        5 kB
        Ahmed Radwan
      8. HADOOP-7754_trunk_rev2.patch
        6 kB
        Ahmed Radwan
      9. HADOOP-7754_branch-1_rev2.patch
        5 kB
        Ahmed Radwan
      10. HADOOP-7754_trunk_rev2.patch
        6 kB
        Ahmed Radwan
      11. HADOOP-7754_trunk.patch
        5 kB
        Ahmed Radwan
      12. hadoop-7754-0.23.0-hasfd.txt
        4 kB
        Cristina L. Abad
      13. hasfd.txt
        4 kB
        Todd Lipcon

        Issue Links

          Activity

          Hide
          Todd Lipcon added a comment -

          In the proof-of-concept patch in HADOOP-7714, I added calls to FSDataInputStream to allow implementors to expose a file descriptor. It's a little bit ugly. If anyone has better ideas how to achieve the same goal, would be very much appreciated.

          Show
          Todd Lipcon added a comment - In the proof-of-concept patch in HADOOP-7714 , I added calls to FSDataInputStream to allow implementors to expose a file descriptor. It's a little bit ugly. If anyone has better ideas how to achieve the same goal, would be very much appreciated.
          Hide
          Todd Lipcon added a comment -

          Here is the 0.20-based "HasFd" patch.

          Show
          Todd Lipcon added a comment - Here is the 0.20-based "HasFd" patch.
          Hide
          Eli Collins added a comment -

          This approach seems reasonable to me. The performance improvement is going to require some implementation level knowledge of the underlying stream, and this seems to be a minimal way to expose it.

          Thinking ahead, we'd like to perform these optimizations on a per-file basis so we'll need a way to map and plumb flags passed when opening the file to blocks (as part of DataXferProtocol) in which case we could use a particular flag, or the presence of particular optimization flags, to indicate whether a stream has a file descriptor, but I think an explicit interface is cleaner.

          Show
          Eli Collins added a comment - This approach seems reasonable to me. The performance improvement is going to require some implementation level knowledge of the underlying stream, and this seems to be a minimal way to expose it. Thinking ahead, we'd like to perform these optimizations on a per-file basis so we'll need a way to map and plumb flags passed when opening the file to blocks (as part of DataXferProtocol) in which case we could use a particular flag, or the presence of particular optimization flags, to indicate whether a stream has a file descriptor, but I think an explicit interface is cleaner.
          Hide
          Cristina L. Abad added a comment -

          Here's a new version of the patch, targeted at 0.23.0.

          Show
          Cristina L. Abad added a comment - Here's a new version of the patch, targeted at 0.23.0.
          Hide
          Ahmed Radwan added a comment -

          Here is an updated patch that applies cleanly to trunk.

          Show
          Ahmed Radwan added a comment - Here is an updated patch that applies cleanly to trunk.
          Hide
          Aaron T. Myers added a comment -

          Marking PA for Ahmed.

          Show
          Aaron T. Myers added a comment - Marking PA for Ahmed.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539251/HADOOP-7754_trunk.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1251//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1251//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539251/HADOOP-7754_trunk.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1251//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1251//console This message is automatically generated.
          Hide
          Ahmed Radwan added a comment -

          One of the new files was missing the license header and docs. Here is an updated versions for both branch-1 and trunk.

          Show
          Ahmed Radwan added a comment - One of the new files was missing the license header and docs. Here is an updated versions for both branch-1 and trunk.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539740/HADOOP-7754_branch-1_rev2.patch
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1260//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539740/HADOOP-7754_branch-1_rev2.patch against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1260//console This message is automatically generated.
          Hide
          Todd Lipcon added a comment -

          Hey Ahmed. Can you upload just the trunk version alone so that test-patch can run? When you upload two at once, it gets confused and seems to be trying to test the branch-1 versoin instead of the trunk one.

          Show
          Todd Lipcon added a comment - Hey Ahmed. Can you upload just the trunk version alone so that test-patch can run? When you upload two at once, it gets confused and seems to be trying to test the branch-1 versoin instead of the trunk one.
          Hide
          Ahmed Radwan added a comment -

          Thanks Todd, Here is the trunk version alone.

          Show
          Ahmed Radwan added a comment - Thanks Todd, Here is the trunk version alone.
          Hide
          Alejandro Abdelnur added a comment -

          I don't see the HasFileDescriptor.java in the patch. The method getFD() should be getFileDescriptor(), other than that looks good. Missing Javadocs.

          Show
          Alejandro Abdelnur added a comment - I don't see the HasFileDescriptor.java in the patch. The method getFD() should be getFileDescriptor(), other than that looks good. Missing Javadocs.
          Hide
          Alejandro Abdelnur added a comment -

          my bad on the missing HasDescriptor.java & javadocs. I was doing an 'svn diff' and didn't show up as I did not 'svn add' it.

          So the only thing is the getFD() renaming.

          Show
          Alejandro Abdelnur added a comment - my bad on the missing HasDescriptor.java & javadocs. I was doing an 'svn diff' and didn't show up as I did not 'svn add' it. So the only thing is the getFD() renaming.
          Hide
          Ahmed Radwan added a comment -

          Thanks Tucu for the review! I think the getFD() naming was to be similar with methods like FileInputStream#getFD(). But, I also see your point in using the full name to be more clear and descriptive, and to also avoid any confusion. I have updated the patched per your comments. Thanks again!

          Show
          Ahmed Radwan added a comment - Thanks Tucu for the review! I think the getFD() naming was to be similar with methods like FileInputStream#getFD(). But, I also see your point in using the full name to be more clear and descriptive, and to also avoid any confusion. I have updated the patched per your comments. Thanks again!
          Hide
          Ahmed Radwan added a comment -

          Here is the branch-1 update.

          Show
          Ahmed Radwan added a comment - Here is the branch-1 update.
          Hide
          Ahmed Radwan added a comment -

          Here is the trunk update.

          Show
          Ahmed Radwan added a comment - Here is the trunk update.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12539932/HADOOP-7754_trunk_rev3.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          -1 tests included. The patch doesn't appear to include any new or modified tests.
          Please justify why no new tests are needed for this patch.
          Also please list what manual steps were performed to verify this patch.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common.

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1268//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1268//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539932/HADOOP-7754_trunk_rev3.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-common-project/hadoop-common. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1268//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1268//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          Looks good. Would be possible to add a simple testcase that asserts that for RawFileSystems you get the FD?

          Show
          Alejandro Abdelnur added a comment - Looks good. Would be possible to add a simple testcase that asserts that for RawFileSystems you get the FD?
          Hide
          Ahmed Radwan added a comment -

          Sure Tucu, I have added such test case. Thanks! Here are the updated patches.

          Show
          Ahmed Radwan added a comment - Sure Tucu, I have added such test case. Thanks! Here are the updated patches.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12540733/HADOOP-7754_trunk_rev4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          -1 javac. The patch appears to cause the build to fail.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1284//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540733/HADOOP-7754_trunk_rev4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. -1 javac. The patch appears to cause the build to fail. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1284//console This message is automatically generated.
          Hide
          Brandon Li added a comment -

          I have no problem applying the patch locally. Both trunk and branch-1 patches look good to me.

          Show
          Brandon Li added a comment - I have no problem applying the patch locally. Both trunk and branch-1 patches look good to me.
          Hide
          Ahmed Radwan added a comment -

          Re uploading the trunk patch to trigger the jenkins test-patch.

          Show
          Ahmed Radwan added a comment - Re uploading the trunk patch to trigger the jenkins test-patch.
          Hide
          Alejandro Abdelnur added a comment -

          +1 pending test-patch run.

          Show
          Alejandro Abdelnur added a comment - +1 pending test-patch run.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12540799/HADOOP-7754_trunk_rev4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.fs.TestS3_LocalFileContextURI
          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.TestLocal_S3FileContextURI
          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1287//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1287//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540799/HADOOP-7754_trunk_rev4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.fs.TestS3_LocalFileContextURI org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.TestLocal_S3FileContextURI org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1287//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1287//console This message is automatically generated.
          Hide
          Ahmed Radwan added a comment -

          I have tried the four tests reports by Jenkins above:

          org.apache.hadoop.fs.TestS3_LocalFileContextURI
          org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract
          org.apache.hadoop.fs.TestLocal_S3FileContextURI
          org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract

          All of them run successfully on my local machine.

          Show
          Ahmed Radwan added a comment - I have tried the four tests reports by Jenkins above: org.apache.hadoop.fs.TestS3_LocalFileContextURI org.apache.hadoop.fs.s3native.TestInMemoryNativeS3FileSystemContract org.apache.hadoop.fs.TestLocal_S3FileContextURI org.apache.hadoop.fs.s3.TestInMemoryS3FileSystemContract All of them run successfully on my local machine.
          Hide
          Ahmed Radwan added a comment -

          Per my previous comment. The test failures reported by Jenkins don't seem to be related to this patch.

          Show
          Ahmed Radwan added a comment - Per my previous comment. The test failures reported by Jenkins don't seem to be related to this patch.
          Hide
          Ahmed Radwan added a comment -

          I am also attaching the output from running test-patch.

          Show
          Ahmed Radwan added a comment - I am also attaching the output from running test-patch.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12540912/test-patch-hadoop-7754.txt
          against trunk revision .

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1294//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540912/test-patch-hadoop-7754.txt against trunk revision . -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1294//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          Ahmed, jenkins precommit job picks up the last file you upload, you uploaded the report of your local run, jenkins tried to apply that as patch and it failed. please re-upload, only, your trunk patch.

          Show
          Alejandro Abdelnur added a comment - Ahmed, jenkins precommit job picks up the last file you upload, you uploaded the report of your local run, jenkins tried to apply that as patch and it failed. please re-upload, only, your trunk patch.
          Hide
          Ahmed Radwan added a comment -

          Thanks Tucu, I have already uploaded the trunk patch earlier and it was picked by the precommit job, see the jenkins report above (https://issues.apache.org/jira/browse/HADOOP-7754?focusedCommentId=13433769&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13433769).

          The precommit result was showing the test failures from HADOOP-8699 (i.e, not related to this ticket). I am re-uploading it again, but since HADOOP-8699 is not pushed, so likely the same tests will fail again.

          Show
          Ahmed Radwan added a comment - Thanks Tucu, I have already uploaded the trunk patch earlier and it was picked by the precommit job, see the jenkins report above ( https://issues.apache.org/jira/browse/HADOOP-7754?focusedCommentId=13433769&page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel#comment-13433769 ). The precommit result was showing the test failures from HADOOP-8699 (i.e, not related to this ticket). I am re-uploading it again, but since HADOOP-8699 is not pushed, so likely the same tests will fail again.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12540984/HADOOP-7754_trunk_rev4.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 1 new or modified test files.

          +1 javac. The applied patch does not increase the total number of javac compiler warnings.

          +1 javadoc. The javadoc tool did not generate any warning messages.

          +1 eclipse:eclipse. The patch built with eclipse:eclipse.

          +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

          +1 release audit. The applied patch does not increase the total number of release audit warnings.

          -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common:

          org.apache.hadoop.ha.TestZKFailoverController

          +1 contrib tests. The patch passed contrib unit tests.

          Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1301//testReport/
          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1301//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12540984/HADOOP-7754_trunk_rev4.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 1 new or modified test files. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. -1 core tests. The patch failed these unit tests in hadoop-common-project/hadoop-common: org.apache.hadoop.ha.TestZKFailoverController +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-HADOOP-Build/1301//testReport/ Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/1301//console This message is automatically generated.
          Hide
          Alejandro Abdelnur added a comment -

          +1. failure seems unrelated.

          Show
          Alejandro Abdelnur added a comment - +1. failure seems unrelated.
          Hide
          Alejandro Abdelnur added a comment -

          Thanks Todd and Ahmed. Committed to trunk and branch-2.

          Show
          Alejandro Abdelnur added a comment - Thanks Todd and Ahmed. Committed to trunk and branch-2.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk-Commit #2644 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2644/)
          HADOOP-7754. Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2644 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2644/ ) HADOOP-7754 . Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          Alejandro Abdelnur added a comment -

          Committed backport to branch-1.

          Show
          Alejandro Abdelnur added a comment - Committed backport to branch-1.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk-Commit #2606 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2606/)
          HADOOP-7754. Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2606 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2606/ ) HADOOP-7754 . Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Common-trunk-Commit #2580 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2580/)
          HADOOP-7754. Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235)

          Result = SUCCESS
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2580 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2580/ ) HADOOP-7754 . Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235) Result = SUCCESS tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Hdfs-trunk #1135 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1135/)
          HADOOP-7754. Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Hdfs-trunk #1135 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk/1135/ ) HADOOP-7754 . Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Hide
          Hudson added a comment -

          Integrated in Hadoop-Mapreduce-trunk #1167 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1167/)
          HADOOP-7754. Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235)

          Result = FAILURE
          tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235
          Files :

          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java
          • /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java
          Show
          Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #1167 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk/1167/ ) HADOOP-7754 . Expose file descriptors from Hadoop-wrapped local FileSystems (todd and ahmed via tucu) (Revision 1373235) Result = FAILURE tucu : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1373235 Files : /hadoop/common/trunk/hadoop-common-project/hadoop-common/CHANGES.txt /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/BufferedFSInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/FSDataInputStream.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/HasFileDescriptor.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/RawLocalFileSystem.java /hadoop/common/trunk/hadoop-common-project/hadoop-common/src/test/java/org/apache/hadoop/fs/TestLocalFileSystem.java

            People

            • Assignee:
              Todd Lipcon
              Reporter:
              Todd Lipcon
            • Votes:
              0 Vote for this issue
              Watchers:
              17 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development