I've gone back to the cases that have motivated this in order to better understand why this change is needed.
The first motivating case is
MAPREDUCE-901, which needs to change the constructor to ReduceContext due to an internal type change. As noted above, user applications should never create context objects, so the constructor could be marked as Private Evolving to solve this problem, couldn't it?
The other motivating case is
MAPREDUCE-372 (chain MapReduce), where various fields of the context need to be replaced with custom versions. The approach described in this comment which is to add a new constructor to ReduceContext (a "pseudo copy constructor"), could again be made Private Evolving to mark its scope. This approach didn't cause any information leakage as far as I can tell (the previous version had to add extra getters, which did cause leakage).
Another approach would be to create package private constructors and have a Private Evolving ContextFactory in the same package if the intent is to keep the constructors further away from user code (in IDE autocomplete, for example). The point is that I think we can avoid the problem of leakage with some changes to visibility annotations.