Uploaded image for project: 'Apache Gobblin'
  1. Apache Gobblin
  2. GOBBLIN-582

Allow file overwrite when copying from task staging to task output for FileAwareInputStreamDataWriter

    XMLWordPrintableJSON

Details

    • Improvement
    • Status: Resolved
    • Major
    • Resolution: Fixed
    • 0.14.0
    • 0.14.0
    • distcp-ng
    • None

    Description

      Currently, we disallow overwriting contents of task output when copying contents from task staging dir to output dir.  This can be a problem when a task fails after the commit step is successful. Subsequent task attempts fail during the commit step as they attempt to copy data from staging to output dir, which results in a job failure. We provide an option to force overwriting contents of the task output directory for such scenarios and let the job succeed.

      Attachments

        Activity

          People

            sv2000 Sudarshan Vasudevan
            sv2000 Sudarshan Vasudevan
            Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: