[HDDS-10832] Client should switch to streaming based on OpenKeySession replication

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 2.0.0, 1.4.1
• Component/s: Ozone Client, s3gateway
• Labels: pull-request-available
• Target Version/s: 1.4.1
• Assignee: Attila Doroszlai
• Reporter: Attila Doroszlai
• Created: 08/May/24 20:00
• Updated: 24/Jul/24 06:40
• Resolved: 22/May/24 08:43

Description

ObjectEndpoint allows Ratis streaming only if bucket replication is not EC. However, it ignores default (global) server-side replication, so it may still end up using Ratis streaming with EC.

Steps to reproduce:

1. Change server-side default replication to EC:

     OZONE-SITE.XML_ozone.server.default.replication.type: EC
     OZONE-SITE.XML_ozone.server.default.replication: RS-3-2-1024k
   

2. Create bucket without setting bucket-level replication

   ozone sh bucket create -l OBJECT_STORE /s3v/bucket2
   

3. Put large enough key to trigger Ratis streaming

   aws s3api put-object --endpoint http://s3g:9878 --bucket bucket2 --body share/ozone/lib/rocksdbjni-7.7.3.jar --key key1
   

Result:

s3g_1       | 2024-05-08 19:51:16,885 [qtp932257672-36] WARN server.HttpChannel: handleException /bucket2/key1 java.io.IOException: java.lang.ClassCastException: class org.apache.hadoop.hdds.scm.ECXceiverClientGrpc cannot be cast to class org.apache.hadoop.hdds.scm.XceiverClientRatis (org.apache.hadoop.hdds.scm.ECXceiverClientGrpc and org.apache.hadoop.hdds.scm.XceiverClientRatis are in unnamed module of loader 'app')
...
s3g_1       | Caused by: java.io.IOException: java.lang.ClassCastException: class org.apache.hadoop.hdds.scm.ECXceiverClientGrpc cannot be cast to class org.apache.hadoop.hdds.scm.XceiverClientRatis 
s3g_1       | 	at org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.handleWrite(KeyDataStreamOutput.java:200)
s3g_1       | 	at org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.write(KeyDataStreamOutput.java:170)
s3g_1       | 	at org.apache.hadoop.ozone.client.io.OzoneDataStreamOutput.write(OzoneDataStreamOutput.java:47)
s3g_1       | 	at org.apache.hadoop.hdds.scm.storage.ByteBufferStreamOutput.write(ByteBufferStreamOutput.java:39)
s3g_1       | 	at org.apache.hadoop.ozone.s3.endpoint.ObjectEndpointStreaming.writeToStreamOutput(ObjectEndpointStreaming.java:153)
s3g_1       | 	at org.apache.hadoop.ozone.s3.endpoint.ObjectEndpointStreaming.putKeyWithStream(ObjectEndpointStreaming.java:108)
s3g_1       | 	at org.apache.hadoop.ozone.s3.endpoint.ObjectEndpointStreaming.put(ObjectEndpointStreaming.java:68)
s3g_1       | 	at org.apache.hadoop.ozone.s3.endpoint.ObjectEndpoint.put(ObjectEndpoint.java:307)
...
s3g_1       | Caused by: java.lang.ClassCastException: class org.apache.hadoop.hdds.scm.ECXceiverClientGrpc cannot be cast to class org.apache.hadoop.hdds.scm.XceiverClientRatis (org.apache.hadoop.hdds.scm.ECXceiverClientGrpc and org.apache.hadoop.hdds.scm.XceiverClientRatis are in unnamed module of loader 'app')
s3g_1       | 	at org.apache.hadoop.hdds.scm.storage.BlockDataStreamOutput.<init>(BlockDataStreamOutput.java:169)
s3g_1       | 	at org.apache.hadoop.ozone.client.io.BlockDataStreamOutputEntry.checkStream(BlockDataStreamOutputEntry.java:99)
s3g_1       | 	at org.apache.hadoop.ozone.client.io.BlockDataStreamOutputEntry.write(BlockDataStreamOutputEntry.java:107)
s3g_1       | 	at org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.writeToDataStreamOutput(KeyDataStreamOutput.java:212)
s3g_1       | 	at org.apache.hadoop.ozone.client.io.KeyDataStreamOutput.handleWrite(KeyDataStreamOutput.java:190)
s3g_1       | 	... 79 more

Issue Links

• links to: GitHub Pull Request #6683