[CHUKWA-155] Job History status arrive out of order causing the status to update incorrectly.

• Type: Bug
• Status: Resolved
• Priority: Critical
• Resolution: Fixed
• Fix Version/s: 0.1.2
• Component/s: Data Collection, Data Processors

Environment

Redhat 5.1, Java 6

• Assignee: Cheng
• Reporter: Eric Yang
• Created: 17/Apr/09 23:19
• Updated: 06/Jun/09 23:06
• Resolved: 22/Apr/09 22:47

Description

Job history contains lines like:

Job JOBID="job_200903310541_1747" JOB_STATUS="RUNNING" .
...
Job JOBID="job_200903310541_1747" FINISH_TIME="1238542231308" JOB_STATUS="SUCCESS" FINISHED_MAPS="1338" FINISHED_REDUCES="760" FAILED_MAPS="78" FAILED_REDUCES="43" COUNTERS="..." .

When pushing the data through collectors and demux, the data can arrive out of order. The database is updated with status "RUNNING" instead of "SUCCESS".

Chukwa Sequence ID can be used to sort out of order data before the data is pumped to database.