[MESOS-3067] Implement a streaming response decoder for events stream

• Type: Task
• Status: Resolved
• Priority: Major
• Resolution: Fixed
• Fix Version/s: 0.24.0
• Epic Link: Scheduler API v1
• Sprint: Twitter Mesos Q3 Sprint 2
• Story Points: 3
• Assignee: Benjamin Mahler
• Reporter: Anand Mazumdar
• Created: 16/Jul/15 18:08
• Updated: 30/May/16 15:57
• Resolved: 24/Jul/15 22:29

Description

We need a streaming response decoder to de-serialize chunks sent from the master on the events stream.

From the HTTP API design doc:
Master encodes each Event in RecordIO format, i.e. a string representation of length of the event in bytes followed by JSON or binary Protobuf (possibly compressed) encoded event.

As of now for getting the basic features right , this is being done in the test-cases:

  auto reader = response.get().reader;
  ASSERT_SOME(reader);

  Future<std::string> eventFuture = reader.get().read();
  AWAIT_READY(eventFuture);

  Event event;
  event.ParseFromString(eventFuture.get());

Two things need to happen:

• We need master to emit events in RecordIO format i.e. event size followed by the serialized event instead of just the serialized events as is the case now.
• The decoder class should then abstract away the logic of reading the response and de-serializing events from the stream.

Ideally, the decoder should work with both "json" and "protobuf" responses.

Issue Links

• relates to: MESOS-3601 Formalize all headers and metadata for HTTP API Event Stream