Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-12091

Issues with directories handling in Swift

    XMLWordPrintableJSON

    Details

    • Type: Improvement
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: None
    • Fix Version/s: None
    • Component/s: fs/swift
    • Labels:
      None
    • Flags:
      Important

      Description

      OpenStack Swift doesn't have notion of directories. In Swift everything is object, stored in some container, that belongs to account.
      Current implementation contains many code that handle directories structure in Swift, in particular functions with code that zero length object is a directory.
      While it might be true for certain cases, there are also many cases where "directory" handling cases problems and highly reduce performance.
      For example, if Swift's container has dozens of objects and one of them has 0 length size, than Swift driver thinks it's a directory and report it to
      upper layer as directory. In consequence, this leads to various exceptions and crashes in client side / upper Hadoop layer.

      The propose of this Jira topic is to make directories handling in driver as an optional and configurable. The driver will behave the same, but there
      will be a configurable option that will disable directories handling and so everything will be objects, even those with 0 length size.

      This will cover cases, where clients doesn't care about directories structures.

        Attachments

          Issue Links

            Activity

              People

              • Assignee:
                gvernik Gil Vernik
                Reporter:
                gvernik Gil Vernik
              • Votes:
                0 Vote for this issue
                Watchers:
                2 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: