[HADOOP-12092] Issues with sub-directories in Swift

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: fs/swift
• Flags: Important
• Assignee: Gil Vernik
• Reporter: Gil Vernik
• Created: 16/Jun/15 08:23
• Updated: 16/Jun/15 10:00

Description

OpenStack swift doesn't have notion of directories or sub-directories. In Swift everything is object, stored in container, that belongs to account.
Openstack Swift allows objects to have delimiters and than users can access and filter those objects using delimiter.

Very good explanation appear here http://docs.rackspace.com/files/api/v1/cf-devguide/content/Pseudo-Hierarchical_Folders_Directories-d1e1580.html

Current driver has many code that create nested directories as zero length objects. While it might be needed for some cases, in general this
is wrong when working with Swift and highly affects performance of the driver.

The goal of this Jira is too make "sub-directories" generation as a configurable option. There will be option that will allow to disable sub-directories generation and this will highly improve the performance.

Example: client perform PUT account/container/a/b/c/d/e/f/g.txt and driver is configured not to use sub-directories in Swift, than only one object "a/b/c/d/e/f/g.txt" will be generated in the container.

Issue Links

• is duplicated by: HADOOP-12091 Issues with directories handling in Swift