Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 3.0.0-alpha1
    • Component/s: documentation
    • Labels:
      None

      Description

      In single cluster setup document, the grep job fails.

      16/06/01 00:21:10 INFO mapreduce.Job: Task Id : attempt_1464707543608_0005_m_000030_2, Status : FAILED
      Error: java.io.FileNotFoundException: Path is not a file: /user/aajisaka/input/shellprofile.d
      	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:78)
      	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:64)
      	at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:174)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1740)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:710)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:381)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:661)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2423)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2419)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2419)
      
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)
      	at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
      	at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
      	at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
      	at org.apache.hadoop.ipc.RemoteException.instantiateException(RemoteException.java:121)
      	at org.apache.hadoop.ipc.RemoteException.unwrapRemoteException(RemoteException.java:88)
      	at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:823)
      	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:810)
      	at org.apache.hadoop.hdfs.DFSClient.getLocatedBlocks(DFSClient.java:799)
      	at org.apache.hadoop.hdfs.DFSClient.open(DFSClient.java:1031)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:288)
      	at org.apache.hadoop.hdfs.DistributedFileSystem$4.doCall(DistributedFileSystem.java:284)
      	at org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
      	at org.apache.hadoop.hdfs.DistributedFileSystem.open(DistributedFileSystem.java:296)
      	at org.apache.hadoop.fs.FileSystem.open(FileSystem.java:785)
      	at org.apache.hadoop.mapreduce.lib.input.LineRecordReader.initialize(LineRecordReader.java:85)
      	at org.apache.hadoop.mapred.MapTask$NewTrackingRecordReader.initialize(MapTask.java:548)
      	at org.apache.hadoop.mapred.MapTask.runNewMapper(MapTask.java:786)
      	at org.apache.hadoop.mapred.MapTask.run(MapTask.java:341)
      	at org.apache.hadoop.mapred.YarnChild$2.run(YarnChild.java:174)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755)
      	at org.apache.hadoop.mapred.YarnChild.main(YarnChild.java:168)
      Caused by: org.apache.hadoop.ipc.RemoteException(java.io.FileNotFoundException): Path is not a file: /user/aajisaka/input/shellprofile.d
      	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:78)
      	at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:64)
      	at org.apache.hadoop.hdfs.server.namenode.FSDirStatAndListingOp.getBlockLocations(FSDirStatAndListingOp.java:174)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1740)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:710)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:381)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:661)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:989)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2423)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2419)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:422)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1755)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2419)
      
      	at org.apache.hadoop.ipc.Client.getRpcResponse(Client.java:1477)
      	at org.apache.hadoop.ipc.Client.call(Client.java:1421)
      	at org.apache.hadoop.ipc.Client.call(Client.java:1338)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Invoker.invoke(ProtobufRpcEngine.java:234)
      	at com.sun.proxy.$Proxy14.getBlockLocations(Unknown Source)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolTranslatorPB.getBlockLocations(ClientNamenodeProtocolTranslatorPB.java:270)
      	at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      	at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
      	at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
      	at java.lang.reflect.Method.invoke(Method.java:498)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invokeMethod(RetryInvocationHandler.java:268)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:193)
      	at org.apache.hadoop.io.retry.RetryInvocationHandler.invoke(RetryInvocationHandler.java:175)
      	at com.sun.proxy.$Proxy15.getBlockLocations(Unknown Source)
      	at org.apache.hadoop.hdfs.DFSClient.callGetBlockLocations(DFSClient.java:821)
      	... 17 more
      

        Issue Links

          Activity

          Hide
          hudson Hudson added a comment -

          SUCCESS: Integrated in Hadoop-trunk-Commit #9901 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9901/)
          HADOOP-13224. Grep job in Single Cluster document fails. (aajisaka) (aajisaka: rev ead61c45da3cb70192d3b9c374bdc83087c992df)

          • hadoop-common-project/hadoop-common/src/site/markdown/SingleCluster.md.vm
          Show
          hudson Hudson added a comment - SUCCESS: Integrated in Hadoop-trunk-Commit #9901 (See https://builds.apache.org/job/Hadoop-trunk-Commit/9901/ ) HADOOP-13224 . Grep job in Single Cluster document fails. (aajisaka) (aajisaka: rev ead61c45da3cb70192d3b9c374bdc83087c992df) hadoop-common-project/hadoop-common/src/site/markdown/SingleCluster.md.vm
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Committed this to trunk. Thanks Daniel and Masatake!

          Show
          ajisakaa Akira Ajisaka added a comment - Committed this to trunk. Thanks Daniel and Masatake!
          Hide
          iwasakims Masatake Iwasaki added a comment -

          +1

          Show
          iwasakims Masatake Iwasaki added a comment - +1
          Hide
          ajisakaa Akira Ajisaka added a comment -

          Thanks Daniel!

          Show
          ajisakaa Akira Ajisaka added a comment - Thanks Daniel!
          Hide
          templedf Daniel Templeton added a comment -

          +1 (non-binding). Thanks for catching and fixing that, Akira Ajisaka!

          Show
          templedf Daniel Templeton added a comment - +1 (non-binding). Thanks for catching and fixing that, Akira Ajisaka !
          Hide
          hadoopqa Hadoop QA added a comment -
          +1 overall



          Vote Subsystem Runtime Comment
          0 reexec 0m 16s Docker mode activated.
          +1 @author 0m 0s The patch does not contain any @author tags.
          +1 mvninstall 6m 31s trunk passed
          +1 mvnsite 0m 55s trunk passed
          +1 mvnsite 0m 50s the patch passed
          +1 whitespace 0m 0s The patch has no whitespace issues.
          +1 asflicense 0m 20s The patch does not generate ASF License warnings.
          9m 6s



          Subsystem Report/Notes
          Docker Image:yetus/hadoop:2c91fd8
          JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807199/HADOOP-13224.01.patch
          JIRA Issue HADOOP-13224
          Optional Tests asflicense mvnsite
          uname Linux c7a81a34b9e1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux
          Build tool maven
          Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh
          git revision trunk / bca31fe
          modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common
          Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9622/console
          Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org

          This message was automatically generated.

          Show
          hadoopqa Hadoop QA added a comment - +1 overall Vote Subsystem Runtime Comment 0 reexec 0m 16s Docker mode activated. +1 @author 0m 0s The patch does not contain any @author tags. +1 mvninstall 6m 31s trunk passed +1 mvnsite 0m 55s trunk passed +1 mvnsite 0m 50s the patch passed +1 whitespace 0m 0s The patch has no whitespace issues. +1 asflicense 0m 20s The patch does not generate ASF License warnings. 9m 6s Subsystem Report/Notes Docker Image:yetus/hadoop:2c91fd8 JIRA Patch URL https://issues.apache.org/jira/secure/attachment/12807199/HADOOP-13224.01.patch JIRA Issue HADOOP-13224 Optional Tests asflicense mvnsite uname Linux c7a81a34b9e1 3.13.0-36-lowlatency #63-Ubuntu SMP PREEMPT Wed Sep 3 21:56:12 UTC 2014 x86_64 x86_64 x86_64 GNU/Linux Build tool maven Personality /testptch/hadoop/patchprocess/precommit/personality/provided.sh git revision trunk / bca31fe modules C: hadoop-common-project/hadoop-common U: hadoop-common-project/hadoop-common Console output https://builds.apache.org/job/PreCommit-HADOOP-Build/9622/console Powered by Apache Yetus 0.4.0-SNAPSHOT http://yetus.apache.org This message was automatically generated.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          v1 patch: Copy only the xml files instead of all the files/directories to fix the failure and make the operation consistent with the standalone operation.

          Show
          ajisakaa Akira Ajisaka added a comment - v1 patch: Copy only the xml files instead of all the files/directories to fix the failure and make the operation consistent with the standalone operation.
          Hide
          ajisakaa Akira Ajisaka added a comment -

          HADOOP-11485 added shellprofile.d directory into etc/hadoop directory, so the job fails. In the document, copy etc/hadoop to hdfs://user/username/input and use hdfs://user/username/input as the input of the grep job.

          Show
          ajisakaa Akira Ajisaka added a comment - HADOOP-11485 added shellprofile.d directory into etc/hadoop directory, so the job fails. In the document, copy etc/hadoop to hdfs://user/username/input and use hdfs://user/username/input as the input of the grep job.

            People

            • Assignee:
              ajisakaa Akira Ajisaka
              Reporter:
              ajisakaa Akira Ajisaka
            • Votes:
              0 Vote for this issue
              Watchers:
              4 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development