As dicussed at the Hadoop Summit and HUG11, we could improve usability a lot simply by shipping with all the required components and manage them all. I see two way of doing this:
- We keep all components separated and start/stop them in the right order. It would be much easier for us to develop but it would be much less friendly to use than the next solution.
- Integrate the DNs and TTs with the RSs, the NN, the JT and a ZK quorum member with the master. I'm not sure if it's easily doable but that's what we do for our unit tests so it must be possible.
Then we need to make sure that we have options to disable that management like we currently do for ZK.
This would also give us the chance to configure HDFS with the right amount of xcievers out of the box, among other settings.