I'm not working on a patch for this at the moment. I wouldn't mind (I'll have to allocate some time for it), but before doing so, I need a clear cut on the APIs that are interface, and they should be in their own JAR file, separate from the JAR file with the implementation classes.
Regarding your comment on 'task code does not load that many libraries', I'm not sure is a valid statement if you are doing Pig or Hive, just check how many more JARs they bring into the classpath, and then check of those JARs how many of them overlap with Hadoop JARs, and then check their versions.
Users get in trouble without making conflicts in a deliberate way, they get into those conflicts because of the lack of isolation of the current model (refer to second paragraph of answer to Mahadev).
Having the use in control as the default is, IMO, the wrong way to go, all the testing you've done on a release becomes meaningless as the users classpath will override any library used by the cluster by default.
My take is that we should fix this via classloader isolation from the get go.
If folks consider this to be a TOO big task (IMO, it will aways be) and the user&system classpath swap is a stop gap alternative, my take is that we should for a default of system-user classpath.
I personally see this as a big issue (using Hadoop from Oozie means that I have to deal with libraries from Hadoop/Pig/Hive/Sqoop/Hbase in the classpath).