This is also somewhat relevant to
I'd like to point out an interesting possibility raised in another JIRA by Joep Rottinghuis. With v.2 and especially early on with v.2, it would be rather useful to be able to enable both v.1 (or v.1.5) and v.2. That would provide a useful verification and comparison environment with a single cluster. The way it's being discussed right now, it sounds like the version would be a single value (mutually exclusive). Wouldn't it be good to have a possibility to be able to enable more than one version? Thoughts?
Although the documentation appears to suggest the use is primarily for clients, I believe it strongly implies it applies to the server-side too. After all, even if it is indicated to the client that the timeline service is enabled, it would be no use obviously (and actually worse than not indicating) if the timeline service was not running. I think it's a completely compatible interpretation it also means strongly that the server-side can interpret this as an instruction to enable all things timeline service.
if the timelineserver daemon is started it directly starts the timelinestore without checking for the configuration "yarn.timeline-service.enabled"
The timelineserver daemon might be able to do it, but the issue is with any other server-side component (e.g. RM, etc.) that needs to see if the timeline server should be supported/used. IMO it should be checked by everything (server-side and client-side) to see if the timeline service is and should be enabled.
I also think ideally each framework (client) should define whether they will use the timeline service. Just because the timeline service is enabled doesn't mean they will use it (like MR today). Ideally the framework should have its own config to use it. Any code of theirs to use the timeline service should be predicated on both properties being true.
Regarding versions, as long as clients can discover that the version they want to use is included in the enabled versions (see above) and if their config is also enabled, it should be able to write/read (provided the APIs exist of course).
IIUC, the newly proposed yarn.timeline-service.version supports a sanity check mechanism: each API should check if the current running ATS's version is equal to or higher than it's required version.
I'm not entirely sure if this is feasible. "Higher" doesn't necessarily mean it will support all versions at and below its own. For example, I don't think we can make a guarantee that v.2 will be able to support all queries for v.1 and v.1.5. What's more appropriate is an exact match. IMO we should not make guarantees about lower version compatibility as it's going to be very challenging to pull off and constraining for the newer implementation.