[OOZIE-2402] oozie-setup.sh sharelib create takes a long time on large clusters

• Type: Improvement
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 4.2.0
• Fix Version/s: 4.3.0
• Component/s: tools
• Assignee: Illya Yalovyy
• Reporter: Illya Yalovyy
• Created: 17/Nov/15 21:20
• Updated: 02/Dec/16 21:02
• Resolved: 15/Dec/15 22:24

Description

When cluster has 256+ nodes it can take up to 5 minutes to create a sharelib.
Copy the tarball itself takes only around 10 seconds. It seems like performance could be improved by loading files concurrently in many threads.