You can indeed shell escape with ! command. No notion of a local working dir, unfortunately. (Next task: lcd and lls commands!)
Currently you can't launch a Hadoop job within this shell. I think that's another patch / separate issue – dealing with the classloader for a jar has enough subtleties associated with it that I punted for the first version. Eventually this shell should be able to do all the hadoop jar, hadoop job, mradmin/dfsadmin etc commands too, but there's a lot of functionality to implement here. All separate tasks in my mind.
In order to establish this code base, my first cut of this system is just focused on implementing filesystem access.
You're right that 'rmr' is a hack. A better argument parsing system is probably necessary moving forward to support things like 'rm -[rf]', 'test', etc. Right now all arguments are treated as filenames in commands like rm; supporting various flags would require using CommandLineParser and other fun. Not impossible, but not first-cut material. Also, jline's tab-completor makes it a bit tricky to handle command-specific -arguments. I'll need to poke around more to figure out if/how that can be done.
As for cd taking a URL – technically it already does. But the current design initializes a FileContext and uses its setWorkingDirectory() method to move around, and then uses FsShell to do some operations like deletes (which require globbing). FsShell doesn't allow reinitialization of the FileSystem, so this code checks for changes in the default FS and reinitializes the structures it uses. We should probably refactor FsShell soon to allow it to work on an arbitrary FileContext instead of the one it instantiates at construction time (or instantiate a new FsShell for each operation? That seems unnecessary). But again, refactoring FsShell I think is beyond the scope of this issue.
All of these are great ideas but I think that they deserve separate tasks. There are enough discussion points as to how to implement each of these that I think a single thread would be confusing. Also getting this committed first would allow different people to attack these problems in parallel if they'd like. I'll file these follow-up tasks after this is resolved. Of course, feel free to file your own in the meantime if you'd like
Thanks for the suggestions!