Details
Description
Problem:
We currently don't have a good user experience around "aurora job
diff" command. The task configs are dumped as "prettified" JSON
strings and diffed with the system diff tool. Anyone who tried to use
it knows it can be very hard to read especially when it comes to
executor data deltas. Also, the implementation is done completely
within the Aurora client making it hard to reuse this feature by other
clients (e.g.: an external deploy coordination tool).Proposal:
Move the diff logic to the scheduler and expose it via a new
jobConfigDiff thrift API.Benefits:
- Client will no longer have the custom non-reusable logic moving us
closer towards a "thin client" goal.- The new RPC can be fully used by any existing or new API clients.
- The diff output will be improved via leveraging third party POJO
and/or JSON diff libraries (1,2,3, etc.).- The server updater can be partially/fully unified with the new diff
logic further improving the overall DRY-ness.Concerns:
- The executor data is currently treated as an opaque string blob on
the scheduler side. In reality, it's almost guaranteed to be JSON. In
order to deliver the best UX, the scheduler would have to start
requiring ExecutorConfig.data to be a valid JSON.(1) -
http://java-object-diff.readthedocs.org/en/latest/getting-started/#getting-started
(2) - http://javers.org/documentation/diff-examples/
(3) - https://github.com/skyscreamer/JSONassert
Full background here: http://markmail.org/message/g7dcqrzjpvc4eyuh
NOTE: the current agreement is to pursue the instance update details (e.g.: added, killed, updated) and add TaskConfig diff details later. See dev thread for more details.