Currently we use the foreach RDD action to submit a spark job. In order to implement job monitoring functionality (
HIVE-7438), we need to get a job identifier when submitting the job, so that we can later register some listener for that specific job.
This task requires facilitation from spark side (
SPARK-2636). I've tried to use AsyncRDDActions instead of the traditional actions, and it proved to be a possible way to get the job ID we need.