[MESOS-3601] Formalize all headers and metadata for HTTP API Event Stream

• Type: Improvement
• Status: Resolved
• Priority: Blocker
• Resolution: Fixed
• Affects Version/s: 0.24.0
• Fix Version/s: 1.2.0
• Labels: api http mesosphere wireprotocol

Environment

Mesos 0.24.0

• Target Version/s: 1.2.0
• Epic Link: Scheduler API v1 improvements
• Sprint: Mesosphere Sprint 49
• Story Points: 5
• Assignee: Anand Mazumdar
• Reporter: Ben Whitehead
• Shepherd: Vinod Kone
• Created: 07/Oct/15 18:37
• Updated: 17/Jan/17 17:26
• Resolved: 17/Jan/17 17:26

Description

From an HTTP standpoint the current set of headers returned when connecting to the HTTP scheduler API are insufficient.

current headers

HTTP/1.1 200 OK
Transfer-Encoding: chunked
Date: Wed, 30 Sep 2015 21:07:16 GMT
Content-Type: application/json

Since the response from mesos is intended to function as a stream Connection: keep-alive should be specified so that the connection can remain open.

If RecordIO is going to be applied to the messages, the headers should include the information necessary for a client to be able to detect RecordIO and setup it response handlers appropriately.

How RecordIO is expressed will come down to the semantics of what is actually "Returned" as the response from POST /api/v1/scheduler.

Proposal

One approach would be to leverage http as much as possible, having a client specify an Accept-Encoding along with the Accept header to indicate that it can handle RecordIO Content-Encoding of Content-Type messages. (This approach allows for things like gzip to be woven in fairly easily in the future)

For this approach I would expect the following:

Request

POST /api/v1/scheduler HTTP/1.1
Host: localhost:5050
Accept: application/x-protobuf
Accept-Encoding: recordio
Content-Type: application/x-protobuf
Content-Length: 35
User-Agent: RxNetty Client

Response

HTTP/1.1 200 OK
Connection: keep-alive
Transfer-Encoding: chunked
Content-Type: application/x-protobuf
Content-Encoding: recordio
Cache-Control: no-transform

When Content-Encoding is used it is recommended to set Cache-Control: no-transform to signal to any proxies that no transformation should be applied to the the content encoding Section 14.11 RFC 2616.

Issue Links

• blocks: MESOS-6876 Default "Accept" type for LAUNCH_NESTED_CONTAINER_SESSION and ATTACH_CONTAINER_OUTPUT should be streaming type
• is related to: MESOS-3067 Implement a streaming response decoder for events stream
• relates to: MESOS-3562 Anomalous bytes in stream from HTTPI Api
• relates to: MESOS-6936 Add support for media types needed for streaming request/responses.
• relates to: MESOS-3566 Add a section to the Scheduler HTTP API docs around RecordIO specification
• links to: Discussion of json stream content-type.