Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Critical Critical
    • Resolution: Fixed
    • Affects Version/s: 0.23.0
    • Fix Version/s: 0.23.1
    • Component/s: viewfs
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      ViewFs.getDelegationTokens returns a list of tokens for the associated namenodes. Credentials serializes these tokens using the service name for the actual namenodes. Effectively, tokens are not cached for viewfs (some more details in MR 3529). Affects any job which uses the TokenCache in tasks along with viewfs (some Pig jobs).

      Talk to Jitendra about this, some options
      1. Change Credentials.getAllTokens to return the key, instead of just a token list (associate the viewfs canonical name with a token in credentials)
      2. Have viewfs issue a fake token.
      Both of these would allow for a single viewfs configuration only.
      3. An additional API in FileSystem - something like getDelegationTokens(String renewer, Credentials credentials) - which would check the credentials object before making token requests to the actual namenode.
      4. An additional API in FileSystem - getCanonicalServiceNames - similar to getDelegationTokens, which would return service names for the actual namenodes. TokenCache/Credentials can work using this list.
      5. have getDelegationTokens check the current UGI - and fetch tokens only if they don't exist.

      Have a quick patch for 3, along with associated MR changes.

      1. HADOOP7933_v2.txt
        11 kB
        Siddharth Seth
      2. HADOOP7933_v1.txt
        11 kB
        Siddharth Seth
      3. HADOOP-7933.txt
        10 kB
        Suresh Srinivas
      4. HDFS2665_v1.txt
        14 kB
        Siddharth Seth
      5. HDFS2665_v1.txt
        14 kB
        Siddharth Seth

        Issue Links

          Activity

          Siddharth Seth created issue -
          Siddharth Seth made changes -
          Field Original Value New Value
          Link This issue blocks MAPREDUCE-3529 [ MAPREDUCE-3529 ]
          Siddharth Seth made changes -
          Attachment HDFS2665_v1.txt [ 12508047 ]
          Siddharth Seth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 0.23.1 [ 12318885 ]
          Siddharth Seth made changes -
          Attachment HDFS2665_v1.txt [ 12508130 ]
          Siddharth Seth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Suresh Srinivas made changes -
          Project Hadoop HDFS [ 12310942 ] Hadoop Common [ 12310240 ]
          Key HDFS-2665 HADOOP-7933
          Affects Version/s 0.23.0 [ 12315569 ]
          Affects Version/s 0.23.0 [ 12315571 ]
          Target Version/s 0.23.1 [ 12318885 ]
          Suresh Srinivas made changes -
          Attachment HADOOP-7933.txt [ 12508137 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 0.23.1 [ 12318884 ]
          Assignee Siddharth Seth [ sseth ]
          Siddharth Seth made changes -
          Attachment HADOOP7933_v1.txt [ 12508168 ]
          Siddharth Seth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Siddharth Seth made changes -
          Link This issue blocks HDFS-2710 [ HDFS-2710 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Siddharth Seth made changes -
          Attachment HADOOP7933_v2.txt [ 12508321 ]
          Siddharth Seth made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Siddharth Seth made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 0.23.1 [ 12318884 ]
          Resolution Fixed [ 1 ]
          Daryn Sharp made changes -
          Link This issue relates to HADOOP-7951 [ HADOOP-7951 ]
          Daryn Sharp made changes -
          Link This issue relates to HADOOP-7967 [ HADOOP-7967 ]
          Arun C Murthy made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Eli Collins made changes -
          Component/s viewfs [ 12318800 ]

            People

            • Assignee:
              Siddharth Seth
              Reporter:
              Siddharth Seth
            • Votes:
              0 Vote for this issue
              Watchers:
              2 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development