[AMBARI-17308] Ambari Logfeeder outputs a lot of errors due to parse date

• Type: Bug
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Affects Version/s: trunk, 2.4.0
• Fix Version/s: 2.4.0

Environment

CentOS7.2, JST

• Assignee: Masahiro Tanaka
• Reporter: Masahiro Tanaka
• Created: 20/Jun/16 06:30
• Updated: 04/Aug/16 11:12
• Resolved: 03/Aug/16 22:13

Description

In logsearch_feeder service log, we got errors like below

2016-06-20 15:28:09,368 ERROR file=ambari-audit.log org.apache.ambari.logfeeder.mapper.MapperDate LogFeederUtil.java:356 - Error applying date transformation. isEpoch=false, dateFormat=yyyy-MM-dd'T'HH:mm:ss.SSSZ, value=2016-06-20T15:28:08.000. mapClass=map_date, input=input:source=file, path=/var/log/ambari-server/ambari-audit.log, fieldName=logtime. Messages suppressed before: 2
java.text.ParseException: Unparseable date: "2016-06-20T15:28:08.000"
	at java.text.DateFormat.parse(DateFormat.java:366)
	at org.apache.ambari.logfeeder.mapper.MapperDate.apply(MapperDate.java:83)
	at org.apache.ambari.logfeeder.filter.Filter.apply(Filter.java:154)
	at org.apache.ambari.logfeeder.filter.FilterGrok.applyMessage(FilterGrok.java:291)
	at org.apache.ambari.logfeeder.filter.FilterGrok.flush(FilterGrok.java:320)
	at org.apache.ambari.logfeeder.input.Input.flush(Input.java:125)
	at org.apache.ambari.logfeeder.input.InputFile.processFile(InputFile.java:430)
	at org.apache.ambari.logfeeder.input.InputFile.start(InputFile.java:260)
	at org.apache.ambari.logfeeder.input.Input.run(Input.java:100)
	at java.lang.Thread.run(Thread.java:745) 

ambari-audit.log is like below

2016-07-21T01:52:49.875+09, User(admin), RemoteIp(192.168.72.1), Operation(Repository update), RequestType(PUT), url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-2.5), ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo id(HDP-2.5), Base URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu14/2.x/BUILDS/2.5.0.0-1025)
2016-07-21T01:52:49.905+09, User(admin), RemoteIp(192.168.72.1), Operation(Repository update), RequestType(PUT), url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu16/repositories/HDP-2.5), ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu16), Repo id(HDP-2.5), Base URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP/ubuntu16/2.x/BUILDS/2.5.0.0-1025)
2016-07-21T01:52:50.015+09, User(admin), RemoteIp(192.168.72.1), Operation(Repository update), RequestType(PUT), url(http://192.168.72.101:8080/api/v1/stacks/HDP/versions/2.5/operating_systems/ubuntu14/repositories/HDP-UTILS-1.1.0.21), ResultStatus(200 OK), Stack(HDP), Stack version(2.5), OS(ubuntu14), Repo id(HDP-UTILS-1.1.0.21), Base URL(http://s3.amazonaws.com/dev.hortonworks.com/HDP-UTILS-1.1.0.21/repos/ubuntu14)

I think date format of the ambari-audit.log (2016-07-21T01:52:49.875+09) should be like 2016-07-21T01:52:49.875+0900, since grok-pattern can't handle 2016-07-21T01:52:49.875+09 format.

Issue Links

• links to: review board