Details
-
Improvement
-
Status: Open
-
Major
-
Resolution: Unresolved
-
None
-
None
-
None
-
None
Description
There is an existing utility "bin/hadoop fs -tail -f <filename>" that prints the last few records from the specified file. A map-reduce application uses a directory as a data-set and it creates multiple files in a HDFS directory. I am proposing that we extend "bin/hadoop fs -tail -f <directory>" to tail the contents of a directory. The files in the directory can be sorted (lexicographically, or based on modtimes) to arrive at the virtual sequence of the set of files inside the directory.