Details
-
Bug
-
Status: Closed
-
Major
-
Resolution: Fixed
-
None
-
None
-
None
-
Flink 1.8.1 with hdfs
Description
After a flink task is over, I get metrics by http request. The record count in response is different with job overview.After a flink task is over, I get metrics by http request. The record count in response is different with job overview.
get http://x.x.x.x:8081/jobs/57969f3978edf3115354fab1a72fd0c8 returns:
{ "jid": "57969f3978edf3115354fab1a72fd0c8", "name": "3349_cjrw_1566177018152", "isStoppable": false, "state": "FINISHED", ... "vertices": [ { "id": "d1cdde18b91ef6ce7c6a1cfdfa9e968d", "name": "CHAIN DataSource (3349_cjrw_1566177018152/SOURCE/0) -> Map (3349_cjrw_1566177018152/AUDIT/0)", "parallelism": 1, "status": "FINISHED", ... "metrics":
... } ... ] }}
But, the metrics by http://x.x.x.x:8081/jobs/57969f3978edf3115354fab1a72fd0c8/vertices/d1cdde18b91ef6ce7c6a1cfdfa9e968d/metrics?get=0.numRecordsOut returns
[\{"id":"0.numRecordsOut","value":"4084803"}]
The overview record count is different with task metrics, please view the apppendix.