Details

    • Type: Sub-task Sub-task
    • Status: Resolved
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 1.0.0
    • Fix Version/s: None
    • Component/s: native
    • Labels:
      None
    • Target Version/s:

      Description

      The initial patch submitted is missing mapping of users to groups. A number of test failures are related to this. The jira tracks adding this support.

      1. HADOOP-8234.branch-1-win.patch
        22 kB
        Bikas Saha
      2. HADOOP-8234.branch-1-win.patch
        11 kB
        Bikas Saha

        Issue Links

          Activity

          Transition Time In Source Status Execution Times Last Executer Last Execution Date
          Open Open Patch Available Patch Available
          5d 36m 1 Bikas Saha 05/Apr/12 00:19
          Patch Available Patch Available Open Open
          1d 19h 13m 1 Sanjay Radia 06/Apr/12 19:33
          Open Open Resolved Resolved
          55d 7h 5m 1 Sanjay Radia 01/Jun/12 02:38
          Chris Nauroth made changes -
          Link This issue is related to HADOOP-8858 [ HADOOP-8858 ]
          Sanjay Radia made changes -
          Status Open [ 1 ] Resolved [ 5 ]
          Target Version/s 1.1.0, 0.24.0 [ 12316501, 12317652 ] HADOOP-1-Windows [ 12320361 ]
          Resolution Fixed [ 1 ]
          Hide
          Sanjay Radia added a comment -

          Thanks Bikas, committed to branch-1-win

          Show
          Sanjay Radia added a comment - Thanks Bikas, committed to branch-1-win
          Bikas Saha made changes -
          Attachment HADOOP-8234.branch-1-win.patch [ 12521712 ]
          Hide
          Bikas Saha added a comment -

          In the old patch I had renamed ShellBasedUnixGroupsMapping to ShellBasedUserGroupsMapping. This patch reverts it since merging will be difficult later on.

          Show
          Bikas Saha added a comment - In the old patch I had renamed ShellBasedUnixGroupsMapping to ShellBasedUserGroupsMapping. This patch reverts it since merging will be difficult later on.
          Hide
          Bikas Saha added a comment -

          The patch does not apply because that works only for trunk.

          I ran the tests on branch-1-win and now 21 additional tests are passing.

            1 org.apache.hadoop.fs.TestFileSystem 
            2 org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser 
            3 org.apache.hadoop.hdfs.security.token.block.TestBlockToken 
            4 org.apache.hadoop.hdfs.server.namenode.TestDFSConcurrentFileOperations 
            5 org.apache.hadoop.hdfs.server.namenode.TestFsck 
            6 org.apache.hadoop.hdfs.TestDFSPermission 
            7 org.apache.hadoop.hdfs.TestDistributedFileSystem 
            8 org.apache.hadoop.hdfs.TestFileAppend4 
            9 org.apache.hadoop.hdfs.TestFileCreation 
           10 org.apache.hadoop.hdfs.TestLeaseRecovery2 
           11 org.apache.hadoop.hdfs.TestQuota 
           12 org.apache.hadoop.hdfs.TestShortCircuitLocalRead 
           13 org.apache.hadoop.hdfs.TestSyncingWriterInterrupted 
           14 org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract 
           15 org.apache.hadoop.mapred.TestMapredSystemDir 
           16 org.apache.hadoop.mapred.TestShuffleJobToken 
           17 org.apache.hadoop.security.TestDistributedCacheAlternateFileSystem 
           18 org.apache.hadoop.security.TestDoAsEffectiveUser 
           19 org.apache.hadoop.security.TestPermission 
           20 org.apache.hadoop.security.TestRefreshUserMappings 
           21 org.apache.hadoop.tools.TestDelegationTokenFetcher
          
          Show
          Bikas Saha added a comment - The patch does not apply because that works only for trunk. I ran the tests on branch-1-win and now 21 additional tests are passing. 1 org.apache.hadoop.fs.TestFileSystem 2 org.apache.hadoop.hdfs.security.TestDelegationTokenForProxyUser 3 org.apache.hadoop.hdfs.security.token.block.TestBlockToken 4 org.apache.hadoop.hdfs.server.namenode.TestDFSConcurrentFileOperations 5 org.apache.hadoop.hdfs.server.namenode.TestFsck 6 org.apache.hadoop.hdfs.TestDFSPermission 7 org.apache.hadoop.hdfs.TestDistributedFileSystem 8 org.apache.hadoop.hdfs.TestFileAppend4 9 org.apache.hadoop.hdfs.TestFileCreation 10 org.apache.hadoop.hdfs.TestLeaseRecovery2 11 org.apache.hadoop.hdfs.TestQuota 12 org.apache.hadoop.hdfs.TestShortCircuitLocalRead 13 org.apache.hadoop.hdfs.TestSyncingWriterInterrupted 14 org.apache.hadoop.hdfs.web.TestWebHdfsFileSystemContract 15 org.apache.hadoop.mapred.TestMapredSystemDir 16 org.apache.hadoop.mapred.TestShuffleJobToken 17 org.apache.hadoop.security.TestDistributedCacheAlternateFileSystem 18 org.apache.hadoop.security.TestDoAsEffectiveUser 19 org.apache.hadoop.security.TestPermission 20 org.apache.hadoop.security.TestRefreshUserMappings 21 org.apache.hadoop.tools.TestDelegationTokenFetcher
          Sanjay Radia made changes -
          Status Patch Available [ 10002 ] Open [ 1 ]
          Target Version/s 0.24.0, 1.1.0 [ 12317652, 12316501 ] 1.1.0, 0.24.0 [ 12316501, 12317652 ]
          Hide
          Bikas Saha added a comment -

          Patch command applied fine on a different source tree on my machine.

          Show
          Bikas Saha added a comment - Patch command applied fine on a different source tree on my machine.
          Hide
          Hadoop QA added a comment -

          -1 overall. Here are the results of testing the latest attachment
          http://issues.apache.org/jira/secure/attachment/12521414/HADOOP-8234.branch-1-win.patch
          against trunk revision .

          +1 @author. The patch does not contain any @author tags.

          +1 tests included. The patch appears to include 15 new or modified tests.

          -1 patch. The patch command could not apply the patch.

          Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/820//console

          This message is automatically generated.

          Show
          Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12521414/HADOOP-8234.branch-1-win.patch against trunk revision . +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 15 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/820//console This message is automatically generated.
          Bikas Saha made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Target Version/s 0.24.0, 1.1.0 [ 12317652, 12316501 ] 1.1.0, 0.24.0 [ 12316501, 12317652 ]
          Bikas Saha made changes -
          Field Original Value New Value
          Attachment HADOOP-8234.branch-1-win.patch [ 12521414 ]
          Hide
          Bikas Saha added a comment -

          posting patch with the shell based user group mappings for windows.
          removing several if(windows) filters because they seem to be running correctly now.

          Show
          Bikas Saha added a comment - posting patch with the shell based user group mappings for windows. removing several if(windows) filters because they seem to be running correctly now.
          Hide
          Bikas Saha added a comment -

          Hadoop has a GroupMappingServiceProvider interface that is used to get group mappings. The commonly used real implementation of that interface is a Shell based mapper that executes shell commands to obtain this information. There has been some work on LDAP and Active Directory based mappers in the community. The Shell based mapper can be made to work on Windows in the following ways
          1) Map every user to the same group, say “HadoopUsers”. Currently, this is what has been hard coded and it could be changed to be a configurable group name.
          2) Use Windows commands like “net user” to provide the group membership of a given user. This provides both local group memberships and global group memberships. I propose we use this method since it’s a natural extension of the existing feature. It also leverages built-in support on Windows for gathering such information.

          Show
          Bikas Saha added a comment - Hadoop has a GroupMappingServiceProvider interface that is used to get group mappings. The commonly used real implementation of that interface is a Shell based mapper that executes shell commands to obtain this information. There has been some work on LDAP and Active Directory based mappers in the community. The Shell based mapper can be made to work on Windows in the following ways 1) Map every user to the same group, say “HadoopUsers”. Currently, this is what has been hard coded and it could be changed to be a configurable group name. 2) Use Windows commands like “net user” to provide the group membership of a given user. This provides both local group memberships and global group memberships. I propose we use this method since it’s a natural extension of the existing feature. It also leverages built-in support on Windows for gathering such information.
          Bikas Saha created issue -

            People

            • Assignee:
              Bikas Saha
              Reporter:
              Bikas Saha
            • Votes:
              0 Vote for this issue
              Watchers:
              3 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development