Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-10129

Distcp may succeed when it fails

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0, 2.0.0-alpha, 3.0.0-alpha1
    • Fix Version/s: 0.23.11, 2.3.0
    • Component/s: tools/distcp
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Distcp uses IOUtils.cleanup to close its output streams w/o previously attempting to close the streams. IOUtils.cleanup will swallow close or implicit flush on close exceptions. As a result, distcp may silently skip files when a partial file listing is generated, and/or appear to succeed when individual copies fail.

        Attachments

        1. HADOOP-10129.patch
          8 kB
          Daryn Sharp

          Activity

            People

            • Assignee:
              daryn Daryn Sharp
              Reporter:
              daryn Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: