Uploaded image for project: 'Hadoop Map/Reduce'
  1. Hadoop Map/Reduce
  2. MAPREDUCE-3825

MR should not be getting duplicate tokens for a MR Job.

    Details

    • Type: Bug
    • Status: Resolved
    • Priority: Major
    • Resolution: Duplicate
    • Affects Version/s: 0.23.1, 2.0.0-alpha
    • Fix Version/s: None
    • Component/s: security
    • Labels:
      None

      Description

      This is the counterpart to HADOOP-7967.
      MR gets tokens for all input, output and the default filesystem when a MR job is submitted.

      The APIs in FileSystem make it challenging to avoid duplicate tokens when there are file systems that have embedded
      filesystems.

      Here is the original description that Daryn wrote:
      The token cache currently tries to assume a filesystem's token service key. The assumption generally worked while there was a one to one mapping of filesystem to token. With the advent of multi-token filesystems like viewfs, the token cache will try to use a service key (ie. for viewfs) that will never exist (because it really gets the mounted fs tokens).

      The descriop

        Attachments

        1. solution4.patch
          13 kB
          Daryn Sharp
        2. TokenCache.pdf
          44 kB
          Daryn Sharp
        3. MAPREDUCE-3825.patch
          8 kB
          Daryn Sharp

          Issue Links

            Activity

              People

              • Assignee:
                daryn Daryn Sharp
                Reporter:
                daryn Daryn Sharp
              • Votes:
                0 Vote for this issue
                Watchers:
                7 Start watching this issue

                Dates

                • Created:
                  Updated:
                  Resolved: