[FLINK-24159] document of entropy injection may mislead users

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 1.14.0, 1.12.5, 1.13.2
• Fix Version/s: 1.14.0, 1.13.3, 1.15.0
• Component/s: Documentation, Runtime / Checkpointing
• Labels: pull-request-available
• Assignee: Feifan Wang
• Reporter: Feifan Wang
• Created: 05/Sep/21 17:41
• Updated: 29/Sep/21 02:36
• Resolved: 22/Sep/21 06:40

Description

FLINK-9061 incroduce entropy inject to s3 path for better scalability, but in document of entropy-injection-for-s3-file-systems use a example with checkpoint directory "s3://my-bucket/checkpoints/entropy/dashboard-job/", with this configuration every checkpoint key will still start with constant checkpoints/ prefix which actually reduces scalability.

Thanks to dmtolpeko for describing this issue in his blog ( flink-and-s3-entropy-injection-for-checkpoints ).

Proposal

alter the checkpoint directory in document of entropy-injection-for-s3-file-systems to "s3://my-bucket/entropy/checkpoints/dashboard-job/" (make entropy key at start of keys).

 

If this proposal is appropriate, I am glad to submit a PR to modify the document here. Any other ideas for this ?

Issue Links

• links to: GitHub Pull Request #17334