Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-19092 ABFS phase 4: post Hadoop 3.4.0 features
  3. HADOOP-17691

Abfs directory delete times out on large directory tree w/ Oauth: OperationTimedOut

    XMLWordPrintableJSON

Details

    • Sub-task
    • Status: Open
    • Major
    • Resolution: Unresolved
    • 3.3.0
    • None
    • fs/azure
    • None

    Description

      Timeouts surfacing on abfs when a delete of a large directory tree is invoked.

      StatusDescription=Operation could not be completed within the specified time.
ErrorCode=OperationTimedOut
ErrorMessage=Operation could not be completed within the specified time.

      This has surfaced in v1 FileOutputCommitter cleanups, implying the directories created there (many many dirs, no files remaining after the job commit) is sufficient to create the problem.

      Attachments

        Issue Links

          is related to

          Bug - A problem which impairs or prevents the functions of the product. HADOOP-17772 wasb: delete() should have timeout option

          • Major - Major loss of function.
          • Open
          relates to

          Sub-task - The sub-task of the issue HADOOP-17687 ABFS: delete call sets Socket timeout lesser than query timeout leading to failures

          • Minor - Minor loss of function, or other problem where easy workaround is present.
          • Open

          New Feature - A new feature of the product, which has yet to be developed. MAPREDUCE-7341 Add a task-manifest output committer for Azure and GCS

          • Major - Major loss of function.
          • Resolved

          Improvement - An improvement or enhancement to an existing feature or task. HADOOP-18013 ABFS: add cloud trash policy with per-schema policy selection

          • Major - Major loss of function.
          • In Progress

          Activity

            People

              Unassigned Unassigned
              stevel@apache.org Steve Loughran
              Votes:
              0 Vote for this issue
              Watchers:
              5 Start watching this issue

              Dates

                Created:
                Updated: