Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-13278

Avoid FileNotFoundException when map/reduce.xml is not available

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Minor
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 2.3.0
    • Component/s: None
    • Labels:
      None
    • Environment:

      Hive on Spark engine
      Found based on :
      Apache Hive 2.0.0
      Apache Spark 1.6.0

      Description

      Many redundant 'File not found' messages appeared in container log during query execution with Hive on Spark.
      Certainly, it doesn't prevent the query from running successfully. So mark it as Minor currently.

      Error message example:

      16/03/14 01:45:06 INFO exec.Utilities: File not found: File does not exist: /tmp/hive/hadoop/2d378538-f5d3-493c-9276-c62dd6634fb4/hive_2016-03-14_01-44-16_835_623058724409492515-6/-mr-10010/0a6d0cae-1eb3-448c-883b-590b3b198a73/reduce.xml
              at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:66)
              at org.apache.hadoop.hdfs.server.namenode.INodeFile.valueOf(INodeFile.java:56)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsUpdateTimes(FSNamesystem.java:1932)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocationsInt(FSNamesystem.java:1873)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1853)
              at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getBlockLocations(FSNamesystem.java:1825)
              at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.getBlockLocations(NameNodeRpcServer.java:565)
              at org.apache.hadoop.hdfs.server.namenode.AuthorizationProviderProxyClientProtocol.getBlockLocations(AuthorizationProviderProxyClientProtocol.java:87)
              at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.getBlockLocations(ClientNamenodeProtocolServerSideTranslatorPB.java:363)
              at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
              at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:617)
              at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:1060)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2086)
              at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2082)
              at java.security.AccessController.doPrivileged(Native Method)
              at javax.security.auth.Subject.doAs(Subject.java:415)
              at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
              at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2080)
      

        Attachments

        1. HIVE-13278.1.patch
          3 kB
          Chao Sun
        2. HIVE-13278.2.patch
          4 kB
          Chao Sun
        3. HIVE-13278.3.patch
          5 kB
          Chao Sun
        4. HIVE-13278.4.patch
          5 kB
          Chao Sun
        5. HIVE-13278.5.patch
          2 kB
          Chao Sun
        6. HIVE-13278.6.patch
          3 kB
          Chao Sun

          Activity

            People

            • Assignee:
              csun Chao Sun
              Reporter:
              xhao1 Xin Hao
            • Votes:
              0 Vote for this issue
              Watchers:
              10 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: