[HDDS-6907] OFS should create buckets with FILE_SYSTEM_OPTIMIZED layout.

• Type: Sub-task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0
• Labels: pull-request-available
• Assignee: Jyotinder Singh Assign to me
• Reporter: Soumitra Sulav
• Created: 17/Jun/22 10:53
• Updated: 21/Jun/22 13:29
• Resolved: 21/Jun/22 13:29

Description

As the config ozone.default.bucket.layout governs the further created bucket layout type, if the default layout type is changed to OBJECT_STORE and the bucket indirectly created via mkdir -p command also results in an OBS bucket that doesn't support further FS operations.

[root@quasar-kykadl-1 ~]# hdfs dfs -mkdir -p ofs://ozone1/vol/buck1
22/06/16 12:23:07 INFO rpc.RpcClient: Creating Bucket: vol/buck1, with om as owner and Versioning false and Storage Type set to DISK and Encryption set to false
[root@quasar-kykadl-1 ~]# ozone sh bucket info vol/buck1
{
  "metadata" : { },
  "volumeName" : "vol",
  "name" : "buck1",
  "storageType" : "DISK",
  "versioning" : false,
  "usedBytes" : 0,
  "usedNamespace" : 0,
  "creationTime" : "2022-06-16T12:23:07.538Z",
  "modificationTime" : "2022-06-16T12:23:07.538Z",
  "quotaInBytes" : -1,
  "quotaInNamespace" : -1,
  "bucketLayout" : "OBJECT_STORE",
  "owner" : "om",
  "link" : false
}
[root@quasar-kykadl-1 ~]# hdfs dfs -ls ofs://ozone1/vol/buck1
-ls: Bucket: buck1 has layout: OBJECT_STORE, which does not support file system semantics. Bucket Layout must be FILE_SYSTEM_OPTIMIZED or LEGACY.
 

Problems that may arise :
In case the user adds a component later after changing the default to OBS, the component's first run of DFS directory creation will be with OBS layout type which doesn't support Filesystem semantics.
Might see issues with Hive/Spark using Ozone FS.

Bucket created from fs cli should always be FSO not depending on the default type.

Issue Links

• links to: GitHub Pull Request #3528