Uploaded image for project: 'Hadoop HDFS'
  1. Hadoop HDFS
  2. HDFS-5335

DFSOutputStream#close() keeps throwing exceptions when it is called multiple times

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Major
    • Resolution: Fixed
    • 2.1.1-beta
    • 2.3.0
    • None
    • None

    Description

      The semantic of DFSOutputStream#close() is incorrect.
      Close() will throw the last exception that it encounters during the flush operations when closing the stream. The exception that it remembers never get cleared therefore spurious exceptions arise when close() is called multiple times.

      Here is the stack trace from the client

      java.nio.channels.ClosedChannelException
      at org.apache.hadoop.hdfs.DFSOutputStream.checkClosed(DFSOutputStream.java:1317)
      at org.apache.hadoop.hdfs.DFSOutputStream.waitForAckedSeqno(DFSOutputStream.java:1810)
      at org.apache.hadoop.hdfs.DFSOutputStream.flushInternal(DFSOutputStream.java:1789)
      at org.apache.hadoop.hdfs.DFSOutputStream.close(DFSOutputStream.java:1877)
      at org.apache.hadoop.fs.FSDataOutputStream$PositionCache.close(FSDataOutputStream.java:71)
      at org.apache.hadoop.fs.FSDataOutputStream.close(FSDataOutputStream.java:104)
      at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:54)
      at org.apache.hadoop.io.IOUtils.copyBytes(IOUtils.java:112)
      at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:366)
      at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:338)
      at org.apache.hadoop.fs.FileUtil.copy(FileUtil.java:289)
      at org.apache.hadoop.mapreduce.JobSubmitter.copyRemoteFiles(JobSubmitter.java:139)
      at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:212)
      at org.apache.hadoop.mapreduce.JobSubmitter.copyAndConfigureFiles(JobSubmitter.java:300)
      at org.apache.hadoop.mapreduce.JobSubmitter.submitJobInternal(JobSubmitter.java:387)
      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1268)
      at org.apache.hadoop.mapreduce.Job$10.run(Job.java:1265)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
      at org.apache.hadoop.mapreduce.Job.submit(Job.java:1265)
      at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:562)
      at org.apache.hadoop.mapred.JobClient$1.run(JobClient.java:557)
      at java.security.AccessController.doPrivileged(Native Method)
      at javax.security.auth.Subject.doAs(Subject.java:396)
      at org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1491)
      at org.apache.hadoop.mapred.JobClient.submitJobInternal(JobClient.java:557)
      at org.apache.hadoop.mapred.JobClient.submitJob(JobClient.java:548)
      at org.apache.hadoop.hive.ql.exec.mr.ExecDriver.execute(ExecDriver.java:425)
      at org.apache.hadoop.hive.ql.exec.mr.MapRedTask.execute(MapRedTask.java:136)
      at org.apache.hadoop.hive.ql.exec.Task.executeTask(Task.java:151)
      at org.apache.hadoop.hive.ql.exec.TaskRunner.runSequential(TaskRunner.java:65)
      at org.apache.hadoop.hive.ql.Driver.launchTask(Driver.java:1414)
      at org.apache.hadoop.hive.ql.Driver.execute(Driver.java:1192)
      at org.apache.hadoop.hive.ql.Driver.runInternal(Driver.java:1020)
      at org.apache.hadoop.hive.ql.Driver.run(Driver.java:888)
      at org.apache.hadoop.hive.cli.CliDriver.processLocalCmd(CliDriver.java:259)
      at org.apache.hadoop.hive.cli.CliDriver.processCmd(CliDriver.java:216)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:413)
      at org.apache.hadoop.hive.cli.CliDriver.processLine(CliDriver.java:348)
      at org.apache.hadoop.hive.cli.CliDriver.processReader(CliDriver.java:446)
      at org.apache.hadoop.hive.cli.CliDriver.processFile(CliDriver.java:456)
      at org.apache.hadoop.hive.cli.CliDriver.executeDriver(CliDriver.java:737)
      at org.apache.hadoop.hive.cli.CliDriver.run(CliDriver.java:675)
      at org.apache.hadoop.hive.cli.CliDriver.main(CliDriver.java:614)
      at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
      at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:39)
      at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:25)
      at java.lang.reflect.Method.invoke(Method.java:597)
      at org.apache.hadoop.util.RunJar.main(RunJar.java:212)
      Job Submission failed with exception 'java.nio.channels.ClosedChannelException(null)'
      FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.mr.MapRedTask
      

      Attachments

        1. HDFS-5335.000.patch
          9 kB
          Haohui Mai
        2. HDFS-5335.001.patch
          9 kB
          Haohui Mai

        Issue Links

          Activity

            People

              wheat9 Haohui Mai
              arpitgupta Arpit Gupta
              Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

                Created:
                Updated:
                Resolved: