This is happening because of the way HOD currently looks for a free port - it tries to connect and if it gets an exception, it takes it as a free port. And this fails while trying with source ports and/or ports in CLOSE_WAIT state as said above.
This patch changes the implementation, by testing for a free port by 'binding' to the port and ensuring that we don't get a bind exception. Tested this behaviour. Also carried tests to make sure that ports bound like this and found to be free are still free and usable by the time the hadoop daemons actually bind to them.
Marked some unused methods, as UNUSED.