This problem was encountered when trying to specify container image nvcr.io/nvidia/tensorflow:19.12-tf1-py3
When initiating Docker Registry authentication (https://docs.docker.com/registry/spec/auth/token/) with nvcr.io, Mesos URI fetcher receives 'WWW-Authenticate' header without 'service' and 'scope' params, and fails here:
This is an example of an unsuccessful request made by Mesos:
At the same time, docker is perfectly capable of pulling this image.
Note that the document "Token Authentication Specification" (https://docs.docker.com/registry/spec/auth/token/), on which the Mesos implementation is based, is vague on the issue of registries that do not provide 'scope'/'service' in WWW-Authenticate header.
What Docker does differently (at the very least, in the case of nvcr.io):
It sends the initial request not to the maniferst/blob URI, but to the repository root URI (http:://nvcr.io/v2 in this case):
To this, it receives response with a "realm" that contains no query arguments:
Then, it composes the scope using the image ref and a hardcoded "pull" action:
(in a full accordance with this spec: https://docs.docker.com/registry/spec/auth/scope/)
and sends the following request to https://nvcr.io/proxy_auth :
(Note that 'push' is absent from the scope)