Details
Minor Description
I'm using spark jobs for processing files and found requirement for getting number of records process after spark submit ran successfully. After analyze it we realized spark using hadoop underline where hadoop is generating the _SUCCESS file as an indicator of the successful completion. I would like to propose requirement for adding number of count with the file name, for example for 100 record processed it would generate like "100_SUCCESS" this will give ability to know how many records get processed on completion of job.