[HAWQ-1560] Default LOG level for "data locality ratio:" events contribute to Hawq Master log bloat

• Type: Improvement
• Status: Open
• Priority: Major
• Resolution: Unresolved
• Component/s: Resource Manager
• Assignee: Yi Jin
• Reporter: Kyle Roberts
• Created: 02/Dec/17 00:57
• Updated: 14/Dec/17 20:13

Description

Issue:

Default LOG level for "data locality ratio:" events contribute to Hawq Master log bloat. Should only be needed when investigating/tuning performance.

Case Scenario:

"data locality ratio:" events can report multiple times and, depending upon the query, many times.

Sample event:

2017-11-30 20:33:39.857251 GMT,"gpadmin","testdb",p280349,th-909919968,"[local]",,2017-11-30 20:06:35 GMT,1630750,con25419,cmd28,seg-1,,,,sx2,"LOG","00000","data locality ratio: 1.000; virtual segment number: 1; different host number: 1; virtual segment number per host(avg/min/max): (1/1/1); segment size(avg/min/max): (1016.000 B/1016 B/1016 B); segment size with penalty(avg/min/max): (1016.000 B/1016 B/1016 B); continuity(avg/min/max): (1.000/1.000/1.000).",,,,,"SQL statement ""select count::float4 from public.test_tbl as Ta where Ta.id is null""","analyze test_tbl;",0,,"cdbdatalocality.c",3372,

For example, Hawq master logs are seen with close to 20% of the file size is from only the "data locality ratio:" events.

Product Enhancement suggestion:

Because these are only needed for performance tuning (or for investigation):

In:

https://github.com/apache/incubator-hawq/blob/master/src/backend/cdb/cdbdatalocality.c#L3372

We can change default LOG level to something like DEBUG1, etc.

Issue Links

• links to: GitHub Pull Request #1318