[BIGTOP-852] Improve HDFS helper script

• Type: Sub-task
• Status: Closed
• Priority: Major
• Resolution: Fixed
• Affects Version/s: 0.6.0
• Fix Version/s: 0.6.0
• Component/s: deployment
• Assignee: Bruno Mahé
• Reporter: Mark Grover
• Created: 21/Feb/13 18:51
• Updated: 01/May/13 02:50
• Resolved: 06/Apr/13 22:19

Description

There are a few things that can be improved regarding the HDFS helper script introduced by BIGTOP-547. Some of these issues have been discussed in BIGTOP-637.

1. The script seems to create user directories for users "root" and "jenkins" but not the current user running the script. I think it will be a good idea to add in the script the commands to create the /user/$USER directory in HDFS. Of course, we should be careful in case the user running the command is root or jenkins, in that case a simple mkdir command would throw error given that those directories already exist.

2. The script uses sudo which requires login shell. However, login shells are not available for certain use cases like init scripts. Consequently, we should consider using su instead.

3. Look into how the helper script can be made faster

Folks, please feel free to add/edit if there is something I missed.

Issue Links

• is related to: BIGTOP-952 init-hdfs.sh is dog slow. Let's replace it with a direct HDFS API calls and better layout management
• is related to: BIGTOP-547 create an HDFS setup helper script