[HDDS-7449] Bucket encryption properties are overwritten when replication config is set

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 1.3.0, 1.4.0
• Component/s: Ozone CLI, Ozone Client
• Labels: pull-request-available
• Assignee: Saketa Chalamchala
• Reporter: Saketa Chalamchala
• Created: 01/Nov/22 20:07
• Updated: 24/Jan/24 07:25
• Resolved: 27/Nov/22 14:02

Description

If we create an encrypted Ozone bucket and then enable EC using set-replication-config, encryption property for the bucket is overwritten and the bucket is not encrypted anymore.

# ozone sh bucket create -k ozKey1 o3://ozone1/testvol/ec-enc-bucket2 --layout FILE_SYSTEM_OPTIMIZED
22/11/01 16:03:30 INFO rpc.RpcClient: Creating Bucket: testvol/ec-enc-bucket2, with the Bucket Layout FILE_SYSTEM_OPTIMIZED, admin as owner, Versioning false, Storage Type set to DISK and Encryption set to true 
# ozone sh bucket set-replication-config o3://ozone1/testvol/ec-enc-bucket2 --type EC --replication rs-3-2-1024k
# ozone sh bucket info o3://ozone1/testvol/ec-enc-bucket2
{
  "metadata" : { },
  "volumeName" : "testvol",
  "name" : "ec-enc-bucket2",
  "storageType" : "DISK",
  "versioning" : false,
  "usedBytes" : 0,
  "usedNamespace" : 0,
  "creationTime" : "2022-11-01T16:03:30.551Z",
  "modificationTime" : "2022-11-01T16:03:44.871Z",
  "quotaInBytes" : -1,
  "quotaInNamespace" : -1,
  "bucketLayout" : "FILE_SYSTEM_OPTIMIZED",
  "link" : false,
  "replicationConfig" : {
    "data" : 3,
    "parity" : 2,
    "ecChunkSize" : 1048576,
    "codec" : "RS",
    "replicationType" : "EC",
    "requiredNodes" : 5
  }
} 

### cdpuser1 does not have any privilege to use ozKey1 encryption key but is allowed to write to the bucket
# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [e-mail]


Valid starting       Expires              Service principal
11/01/2022 12:41:00  11/02/2022 12:41:00  krbtgt/[e-mail]
	renew until 11/09/2022 11:41:00
# hdfs dfs -put test.txt ofs://ozone1/testvol/ec-enc-bucket2/test2.txt
22/11/01 12:41:14 WARN erasurecode.ErasureCodeNative: Loading ISA-L failed: Failed to load libisal.so.2 (libisal.so.2: cannot open shared object file: No such file or directory)
22/11/01 12:41:14 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable
22/11/01 12:41:14 WARN impl.MetricsConfig: Cannot locate configuration: tried hadoop-metrics2-xceiverclientmetrics.properties,hadoop-metrics2.properties
22/11/01 12:41:14 INFO impl.MetricsSystemImpl: Scheduled Metric snapshot period at 10 second(s).
22/11/01 12:41:14 INFO impl.MetricsSystemImpl: XceiverClientMetrics metrics system started
# hdfs dfs -ls ofs://ozone1/testvol/ec-enc-bucket2
Found 3 items
...
-rw-rw-rw-   5 cdpuser1 cdpuser1          8 2022-11-01 12:41 ofs://ozone1/testvol/ec-enc-bucket2/test2.txt 

If both encryption key and replication config are specified at bucket creation time, everything works as expected.

# ozone sh bucket create -k ozKey1 o3://ozone1/testvol/ec-enc-bucket --type EC --replication rs-3-2-1024k --layout FILE_SYSTEM_OPTIMIZED
22/11/01 16:02:31 INFO rpc.RpcClient: Creating Bucket: testvol/ec-enc-bucket, with the Bucket Layout FILE_SYSTEM_OPTIMIZED, admin as owner, Versioning false, Storage Type set to DISK and Encryption set to true 
# ozone sh bucket info o3://ozone1/testvol/ec-enc-bucket
{
  "metadata" : { },
  "volumeName" : "testvol",
  "name" : "ec-enc-bucket",
  "storageType" : "DISK",
  "versioning" : false,
  "usedBytes" : 0,
  "usedNamespace" : 0,
  "creationTime" : "2022-11-01T16:02:31.476Z",
  "modificationTime" : "2022-11-01T16:02:31.476Z",
  "encryptionKeyName" : "ozKey1",
  "quotaInBytes" : -1,
  "quotaInNamespace" : -1,
  "bucketLayout" : "FILE_SYSTEM_OPTIMIZED",
  "owner" : "admin",
  "link" : false,
  "replicationConfig" : {
    "data" : 3,
    "parity" : 2,
    "ecChunkSize" : 1048576,
    "codec" : "RS",
    "replicationType" : "EC",
    "requiredNodes" : 5
  }
} 

### cdpuser1 does not have any privilege to use ozKey1 encryption key and is not allowed to write to the bucket
#  kinit cdpuser1
Password for [e-mail]: 
[root@ccycloud-1 ~]# klist
Ticket cache: FILE:/tmp/krb5cc_0
Default principal: [e-mail]


Valid starting       Expires              Service principal
11/01/2022 12:41:00  11/02/2022 12:41:00  krbtgt/[e-mail]
	renew until 11/09/2022 11:41:00

# hdfs dfs -put test.txt ofs://ozone1/testvol/ec-enc-bucket/test2.txt
22/11/01 12:41:22 WARN erasurecode.ErasureCodeNative: Loading ISA-L failed: Failed to load libisal.so.2 (libisal.so.2: cannot open shared object file: No such file or directory)
22/11/01 12:41:22 WARN erasurecode.ErasureCodeNative: ISA-L support is not available in your platform... using builtin-java codec where applicable
put: User:cdpuser1 not allowed to do 'DECRYPT_EEK' on 'ozKey1'

 

Issue Links

• relates to: HDDS-10142 Add hidden command to set bucket encryption key to fix HDDS-7449
• links to: GitHub Pull Request #3992