Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3782

teragen terasort jobs fail when using webhdfs://

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.1, 0.24.0
    • Fix Version/s: 0.23.3, 2.0.2-alpha
    • Component/s: mrv2
    • Labels:
      None

      Description

      When running a teragen job with a webhdfs:// url the delegation token that is retrieved is an hdfs delegation token.

      And the subsequent terasort job on the output fails with java io exception

        Activity

        Hide
        Arpit Gupta added a comment -

        here are the steps to reproduce the issue (make sure webhdfs is enabled)

        1. run teragen

        bin/hadoop --config HADOOP_CONF_DIR jar hadoop-mapreduce-examples-0.23.1-SNAPSHOT.jar teragen 10000 webhdfs://NN:50070/user/path

        prints the following

        12/02/01 20:03:33 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 158 for hrt_qa on 98.137.234.231:8020
        12/02/01 20:03:33 INFO security.TokenCache: Got dt for hdfs://NN:8020;uri=IP:8020;t.service=IP:8020
        

        Since a webhdfs url was provided a webhdfs delegation token should have been used.

        2. Run terasort

        bin/hadoop --config HADOOP_CONF_DIR jar hadoop-mapreduce-examples-0.23.1-SNAPSHOT.jar terasort webhdfs://NN:50070/user/path webhdfs://NN:50070/user/path2

        This gets 2 delegation tokens one webhdfs and the other hdfs

        12/02/01 20:03:48 INFO terasort.TeraSort: starting
        12/02/01 20:03:49 INFO security.TokenCache: Got dt for webhdfs://NN:50070;uri=IP:50070;t.service=IP:50070
        Spent 65ms computing base-splits.
        Spent 2ms computing TeraScheduler splits.
        Computing input splits took 67ms
        Sampling 2 splits of 2
        Making 1 from 10000 sampled records
        Computing parititions took 1668ms
        Spent 1740ms computing partitions.
        12/02/01 20:03:51 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 161 for USER on 98.137.234.231:8020
        12/02/01 20:03:51 INFO security.TokenCache: Got dt for hdfs://NN:8020;uri=IP:8020;t.service=IP:8020
        

        Both the tokens should be webhdfs delegation tokens.

        Then we see a java io exception and the job fails. Below is the stack trace

        2/02/01 20:03:54 INFO mapreduce.Job: Job job_1328054538421_0037 failed with state FAILED due to: Application application_1328054538421_0037 failed 1 times due to AM Container for appattempt_1328054538421_0037_000001 exited with  exitCode: -1000 due to: RemoteTrace: 
        java.io.IOException: Offset=0 out of the range [0, 0); OPEN, path=/path/terajobs/output/_partition.lst
                at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:167)
                at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:267)
                at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$000(WebHdfsFileSystem.java:105)
                at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$OffsetUrlInputStream.checkResponseCode(WebHdfsFileSystem.java:676)
                at org.apache.hadoop.hdfs.ByteRangeInputStream.getInputStream(ByteRangeInputStream.java:106)
                at org.apache.hadoop.hdfs.ByteRangeInputStream.read(ByteRangeInputStream.java:130)
                at java.io.InputStream.read(InputStream.java:154)
                at java.io.DataInputStream.read(DataInputStream.java:83)
                at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:75)
                at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:49)
                at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:109)
                at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:260)
                at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232)
                at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:183)
                at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1837)
                at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1806)
                at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1782)
                at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:95)
                at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:49)
                at org.apache.hadoop.yarn.util.FSDownload$1.run(FSDownload.java:157)
                at org.apache.hadoop.yarn.util.FSDownload$1.run(FSDownload.java:155)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:396)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
                at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:153)
                at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:49)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
                at java.util.concurrent.FutureTask.run(FutureTask.java:138)
                at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441)
                at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303)
                at java.util.concurrent.FutureTask.run(FutureTask.java:138)
                at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886)
                at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908)
                at java.lang.Thread.run(Thread.java:662)
         at LocalTrace: 
                org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl: Offset=0 out of the range [0, 0); OPEN, path=/user/hrt_qa/hdfsRegressionData/terajobs/output/_partition.lst
                at org.apache.hadoop.yarn.server.nodemanager.api.protocolrecords.impl.pb.LocalResourceStatusPBImpl.convertFromProtoFormat(LocalResourceStatusPBImpl.java:217)
                at org.apache.hadoop.yarn.server.nodemanager.api.protocolrecords.impl.pb.LocalResourceStatusPBImpl.getException(LocalResourceStatusPBImpl.java:147)
                at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.update(ResourceLocalizationService.java:827)
                at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.processHeartbeat(ResourceLocalizationService.java:497)
                at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.heartbeat(ResourceLocalizationService.java:222)
                at org.apache.hadoop.yarn.server.nodemanager.api.impl.pb.service.LocalizationProtocolPBServiceImpl.heartbeat(LocalizationProtocolPBServiceImpl.java:46)
                at org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2.callBlockingMethod(LocalizationProtocol.java:57)
                at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Server.call(ProtoOverHadoopRpcEngine.java:342)
                at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1493)
                at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1489)
                at java.security.AccessController.doPrivileged(Native Method)
                at javax.security.auth.Subject.doAs(Subject.java:396)
                at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177)
                at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1487)
        
        Show
        Arpit Gupta added a comment - here are the steps to reproduce the issue (make sure webhdfs is enabled) 1. run teragen bin/hadoop --config HADOOP_CONF_DIR jar hadoop-mapreduce-examples-0.23.1-SNAPSHOT.jar teragen 10000 webhdfs://NN:50070/user/path prints the following 12/02/01 20:03:33 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 158 for hrt_qa on 98.137.234.231:8020 12/02/01 20:03:33 INFO security.TokenCache: Got dt for hdfs: //NN:8020;uri=IP:8020;t.service=IP:8020 Since a webhdfs url was provided a webhdfs delegation token should have been used. 2. Run terasort bin/hadoop --config HADOOP_CONF_DIR jar hadoop-mapreduce-examples-0.23.1-SNAPSHOT.jar terasort webhdfs://NN:50070/user/path webhdfs://NN:50070/user/path2 This gets 2 delegation tokens one webhdfs and the other hdfs 12/02/01 20:03:48 INFO terasort.TeraSort: starting 12/02/01 20:03:49 INFO security.TokenCache: Got dt for webhdfs: //NN:50070;uri=IP:50070;t.service=IP:50070 Spent 65ms computing base-splits. Spent 2ms computing TeraScheduler splits. Computing input splits took 67ms Sampling 2 splits of 2 Making 1 from 10000 sampled records Computing parititions took 1668ms Spent 1740ms computing partitions. 12/02/01 20:03:51 INFO hdfs.DFSClient: Created HDFS_DELEGATION_TOKEN token 161 for USER on 98.137.234.231:8020 12/02/01 20:03:51 INFO security.TokenCache: Got dt for hdfs: //NN:8020;uri=IP:8020;t.service=IP:8020 Both the tokens should be webhdfs delegation tokens. Then we see a java io exception and the job fails. Below is the stack trace 2/02/01 20:03:54 INFO mapreduce.Job: Job job_1328054538421_0037 failed with state FAILED due to: Application application_1328054538421_0037 failed 1 times due to AM Container for appattempt_1328054538421_0037_000001 exited with exitCode: -1000 due to: RemoteTrace: java.io.IOException: Offset=0 out of the range [0, 0); OPEN, path=/path/terajobs/output/_partition.lst at org.apache.hadoop.hdfs.web.JsonUtil.toRemoteException(JsonUtil.java:167) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.validateResponse(WebHdfsFileSystem.java:267) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem.access$000(WebHdfsFileSystem.java:105) at org.apache.hadoop.hdfs.web.WebHdfsFileSystem$OffsetUrlInputStream.checkResponseCode(WebHdfsFileSystem.java:676) at org.apache.hadoop.hdfs.ByteRangeInputStream.getInputStream(ByteRangeInputStream.java:106) at org.apache.hadoop.hdfs.ByteRangeInputStream.read(ByteRangeInputStream.java:130) at java.io.InputStream.read(InputStream.java:154) at java.io.DataInputStream.read(DataInputStream.java:83) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:75) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:49) at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:109) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:260) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:232) at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:183) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1837) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1806) at org.apache.hadoop.fs.FileSystem.copyToLocalFile(FileSystem.java:1782) at org.apache.hadoop.yarn.util.FSDownload.copy(FSDownload.java:95) at org.apache.hadoop.yarn.util.FSDownload.access$000(FSDownload.java:49) at org.apache.hadoop.yarn.util.FSDownload$1.run(FSDownload.java:157) at org.apache.hadoop.yarn.util.FSDownload$1.run(FSDownload.java:155) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:153) at org.apache.hadoop.yarn.util.FSDownload.call(FSDownload.java:49) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:441) at java.util.concurrent.FutureTask$Sync.innerRun(FutureTask.java:303) at java.util.concurrent.FutureTask.run(FutureTask.java:138) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask(ThreadPoolExecutor.java:886) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:908) at java.lang. Thread .run( Thread .java:662) at LocalTrace: org.apache.hadoop.yarn.exceptions.impl.pb.YarnRemoteExceptionPBImpl: Offset=0 out of the range [0, 0); OPEN, path=/user/hrt_qa/hdfsRegressionData/terajobs/output/_partition.lst at org.apache.hadoop.yarn.server.nodemanager.api.protocolrecords.impl.pb.LocalResourceStatusPBImpl.convertFromProtoFormat(LocalResourceStatusPBImpl.java:217) at org.apache.hadoop.yarn.server.nodemanager.api.protocolrecords.impl.pb.LocalResourceStatusPBImpl.getException(LocalResourceStatusPBImpl.java:147) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerRunner.update(ResourceLocalizationService.java:827) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService$LocalizerTracker.processHeartbeat(ResourceLocalizationService.java:497) at org.apache.hadoop.yarn.server.nodemanager.containermanager.localizer.ResourceLocalizationService.heartbeat(ResourceLocalizationService.java:222) at org.apache.hadoop.yarn.server.nodemanager.api.impl.pb.service.LocalizationProtocolPBServiceImpl.heartbeat(LocalizationProtocolPBServiceImpl.java:46) at org.apache.hadoop.yarn.proto.LocalizationProtocol$LocalizationProtocolService$2.callBlockingMethod(LocalizationProtocol.java:57) at org.apache.hadoop.yarn.ipc.ProtoOverHadoopRpcEngine$Server.call(ProtoOverHadoopRpcEngine.java:342) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1493) at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:1489) at java.security.AccessController.doPrivileged(Native Method) at javax.security.auth.Subject.doAs(Subject.java:396) at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1177) at org.apache.hadoop.ipc.Server$Handler.run(Server.java:1487)
        Hide
        Daryn Sharp added a comment -

        Teragen: Notice it's the DFClient reporting that it got the token. It shouldn't be getting a token via RPC at all. I think it's getting a token for the default fs (hdfs), not the webhdfs path on the cmdline. It "works" because they happen to be the same host and the RPC port isn't blocked.

        Terasort: Again, DFSClient shouldn't be getting the token at all.

        Task failure: Awhile back, I think Nicholas fixed an issue with 0 not being considered valid if the file length is zero. This may no longer be an issue.

        Show
        Daryn Sharp added a comment - Teragen: Notice it's the DFClient reporting that it got the token. It shouldn't be getting a token via RPC at all. I think it's getting a token for the default fs (hdfs), not the webhdfs path on the cmdline. It "works" because they happen to be the same host and the RPC port isn't blocked. Terasort: Again, DFSClient shouldn't be getting the token at all. Task failure: Awhile back, I think Nicholas fixed an issue with 0 not being considered valid if the file length is zero. This may no longer be an issue.
        Hide
        Jason Lowe added a comment -

        Took a closer look at this. The log message about getting a delegation token for HDFS is not in error, as that token is needed to setup the staging directory for the job. So that is working as-expected.

        Teragen failing to get a delegation token for webhdfs is a bug in TeraOutputFormat, as it is responsible for grabbing any necessary delegation tokens in the checkOutputSpecs method.

        Terasort grabs two tokens, one from webhdfs (for the input files and would also for the output directory once TeraOutputFormat is fixed) and one from HDFS for the job staging area. We won't see two delegation token messages for the same filesystem since it doesn't grab a new token if it already has one.

        The range error is a known problem that has been already fixed, see HDFS-3101.

        So to sum up, I think there's only one remaining bug here: TeraOutputFormat is not grabbing a delegation token for the output directory when checkOutputSpecs is called.

        Show
        Jason Lowe added a comment - Took a closer look at this. The log message about getting a delegation token for HDFS is not in error, as that token is needed to setup the staging directory for the job. So that is working as-expected. Teragen failing to get a delegation token for webhdfs is a bug in TeraOutputFormat, as it is responsible for grabbing any necessary delegation tokens in the checkOutputSpecs method. Terasort grabs two tokens, one from webhdfs (for the input files and would also for the output directory once TeraOutputFormat is fixed) and one from HDFS for the job staging area. We won't see two delegation token messages for the same filesystem since it doesn't grab a new token if it already has one. The range error is a known problem that has been already fixed, see HDFS-3101 . So to sum up, I think there's only one remaining bug here: TeraOutputFormat is not grabbing a delegation token for the output directory when checkOutputSpecs is called.
        Hide
        Jason Lowe added a comment -

        Patch for TeraOutputFormat delegation token fix. No automated testcase, but I did manually test on a secure cluster and verified that teragen obtains the delegation token for webhdfs.

        Show
        Jason Lowe added a comment - Patch for TeraOutputFormat delegation token fix. No automated testcase, but I did manually test on a secure cluster and verified that teragen obtains the delegation token for webhdfs.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12539702/MAPREDUCE-3782.patch
        against trunk revision .

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no new tests are needed for this patch.
        Also please list what manual steps were performed to verify this patch.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 eclipse:eclipse. The patch built with eclipse:eclipse.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2716//testReport/
        Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2716//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12539702/MAPREDUCE-3782.patch against trunk revision . +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no new tests are needed for this patch. Also please list what manual steps were performed to verify this patch. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 javadoc. The javadoc tool did not generate any warning messages. +1 eclipse:eclipse. The patch built with eclipse:eclipse. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed unit tests in hadoop-mapreduce-project/hadoop-mapreduce-examples. +1 contrib tests. The patch passed contrib unit tests. Test results: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2716//testReport/ Console output: https://builds.apache.org/job/PreCommit-MAPREDUCE-Build/2716//console This message is automatically generated.
        Hide
        Daryn Sharp added a comment -

        +1 I'm not fond of how formatters have to explicitly obtain tokens, but this fits the current paradigm. Thanks Jason!

        Show
        Daryn Sharp added a comment - +1 I'm not fond of how formatters have to explicitly obtain tokens, but this fits the current paradigm. Thanks Jason!
        Hide
        Robert Joseph Evans added a comment -

        I am +1 too, I'll check this in.

        Show
        Robert Joseph Evans added a comment - I am +1 too, I'll check this in.
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Common-trunk-Commit #2567 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2567/)
        MAPREDUCE-3782. teragen terasort jobs fail when using webhdfs:// (Jason Lowe via bobby) (Revision 1371325)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1371325
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
        Show
        Hudson added a comment - Integrated in Hadoop-Common-trunk-Commit #2567 (See https://builds.apache.org/job/Hadoop-Common-trunk-Commit/2567/ ) MAPREDUCE-3782 . teragen terasort jobs fail when using webhdfs:// (Jason Lowe via bobby) (Revision 1371325) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1371325 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Hdfs-trunk-Commit #2632 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2632/)
        MAPREDUCE-3782. teragen terasort jobs fail when using webhdfs:// (Jason Lowe via bobby) (Revision 1371325)

        Result = SUCCESS
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1371325
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
        Show
        Hudson added a comment - Integrated in Hadoop-Hdfs-trunk-Commit #2632 (See https://builds.apache.org/job/Hadoop-Hdfs-trunk-Commit/2632/ ) MAPREDUCE-3782 . teragen terasort jobs fail when using webhdfs:// (Jason Lowe via bobby) (Revision 1371325) Result = SUCCESS bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1371325 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
        Hide
        Robert Joseph Evans added a comment -

        Thanks Jason, I put this into trunk, branch-2, branch-2.1.0-alpha and branch-0.23

        Show
        Robert Joseph Evans added a comment - Thanks Jason, I put this into trunk, branch-2, branch-2.1.0-alpha and branch-0.23
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #2587 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2587/)
        MAPREDUCE-3782. teragen terasort jobs fail when using webhdfs:// (Jason Lowe via bobby) (Revision 1371325)

        Result = FAILURE
        bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1371325
        Files :

        • /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt
        • /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java
        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #2587 (See https://builds.apache.org/job/Hadoop-Mapreduce-trunk-Commit/2587/ ) MAPREDUCE-3782 . teragen terasort jobs fail when using webhdfs:// (Jason Lowe via bobby) (Revision 1371325) Result = FAILURE bobby : http://svn.apache.org/viewcvs.cgi/?root=Apache-SVN&view=rev&rev=1371325 Files : /hadoop/common/trunk/hadoop-mapreduce-project/CHANGES.txt /hadoop/common/trunk/hadoop-mapreduce-project/hadoop-mapreduce-examples/src/main/java/org/apache/hadoop/examples/terasort/TeraOutputFormat.java

          People

          • Assignee:
            Jason Lowe
            Reporter:
            Arpit Gupta
          • Votes:
            0 Vote for this issue
            Watchers:
            5 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development