Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3825

MR should not be getting duplicate tokens for a MR Job.

    Details

    • Type: Bug Bug
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Duplicate
    • Affects Version/s: 0.23.1, 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None

      Description

      This is the counterpart to HADOOP-7967.
      MR gets tokens for all input, output and the default filesystem when a MR job is submitted.

      The APIs in FileSystem make it challenging to avoid duplicate tokens when there are file systems that have embedded
      filesystems.

      Here is the original description that Daryn wrote:
      The token cache currently tries to assume a filesystem's token service key. The assumption generally worked while there was a one to one mapping of filesystem to token. With the advent of multi-token filesystems like viewfs, the token cache will try to use a service key (ie. for viewfs) that will never exist (because it really gets the mounted fs tokens).

      The descriop

      1. MAPREDUCE-3825.patch
        8 kB
        Daryn Sharp
      2. solution4.patch
        13 kB
        Daryn Sharp
      3. TokenCache.pdf
        44 kB
        Daryn Sharp

        Issue Links

          Activity

          No work has yet been logged on this issue.

            People

            • Assignee:
              Daryn Sharp
              Reporter:
              Daryn Sharp
            • Votes:
              0 Vote for this issue
              Watchers:
              7 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development