Uploaded image for project: 'Hive'
  1. Hive
  2. HIVE-14690

Query fail when hive.exec.parallel=true, with conflicting session dir

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 1.3.0, 2.1.0
    • Fix Version/s: 1.3.0, 2.2.0
    • Component/s: None
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      This happens when hive.scratchdir.lock=true. Error message:

      /hive/scratch/343hdirdp/cab907fc-5e1d-4d69-aa72-d7b442495c7a/inuse.info (inode 19537): File does not exist. [Lease.  Holder: DFSClient_NONMAPREDUCE_1572639975_1, pendingcreates: 2]
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.checkLease(FSNamesystem.java:3430)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.analyzeFileState(FSNamesystem.java:3235)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getNewBlockTargets(FSNamesystem.java:3073)
      	at org.apache.hadoop.hdfs.server.namenode.FSNamesystem.getAdditionalBlock(FSNamesystem.java:3033)
      	at org.apache.hadoop.hdfs.server.namenode.NameNodeRpcServer.addBlock(NameNodeRpcServer.java:725)
      	at org.apache.hadoop.hdfs.protocolPB.ClientNamenodeProtocolServerSideTranslatorPB.addBlock(ClientNamenodeProtocolServerSideTranslatorPB.java:492)
      	at org.apache.hadoop.hdfs.protocol.proto.ClientNamenodeProtocolProtos$ClientNamenodeProtocol$2.callBlockingMethod(ClientNamenodeProtocolProtos.java)
      	at org.apache.hadoop.ipc.ProtobufRpcEngine$Server$ProtoBufRpcInvoker.call(ProtobufRpcEngine.java:616)
      	at org.apache.hadoop.ipc.RPC$Server.call(RPC.java:969)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2137)
      	at org.apache.hadoop.ipc.Server$Handler$1.run(Server.java:2133)
      	at java.security.AccessController.doPrivileged(Native Method)
      	at javax.security.auth.Subject.doAs(Subject.java:415)
      	at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1668)
      	at org.apache.hadoop.ipc.Server$Handler.run(Server.java:2131)
      
      	at org.apache.hadoop.hive.ql.session.SessionState.start(SessionState.java:535)
      	at org.apache.hadoop.hive.ql.exec.TaskRunner.run(TaskRunner.java:74)
      

        Attachments

        1. HIVE-14690.1.patch
          2 kB
          Daniel Dai
        2. HIVE-14690.2.patch
          1 kB
          Daniel Dai
        3. HIVE-14690.3.patch
          1 kB
          Daniel Dai

          Issue Links

            Activity

              People

              • Assignee:
                daijy Daniel Dai
                Reporter:
                daijy Daniel Dai
              • Votes:
                0 Vote for this issue
                Watchers:
                5 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: