[OAK-10418] Query : Faster binaries migration to azure cloud while using OAK repo

• Type: Documentation
• Status: Open
• Priority: Minor
• Resolution: Unresolved
• Component/s: jackrabbit-api
• Assignee: Thomas Mueller
• Reporter: Bharat Wadhwa
• Created: 25/Aug/23 10:25
• Updated: 25/Aug/23 11:33

Description

The Content Transfer Tool is a tool developed by Adobe that you can use to initiate the migration of existing content from a source AEM instance (on-premise or AMS) to the target AEM Cloud Service instance.

For migrating inline blobs, we use the OAK API, which helps upload this data to an Azure container. As per the current architecture, it spawns 20 threads, with each thread uploading one blob at a time. [Repo link|http://example.com]

However, uploading one blob at a time introduces network latency with each call, ultimately increasing the overall migration time.

Proposed solution: Rather than uploading the blobs one at a time, we would store them in temporary storage. Once that step is complete, we can utilize the publicly available azCopy feature, which significantly accelerates the data migration process.

I conducted a Proof of Concept (POC) locally, and we observed a 15% improvement with this approach.