Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-2416

In Gridmix, in RoundRobinUserResolver, the list of groups for a user obtained from users-list-file is incorrect

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.23.0
    • Component/s: contrib/gridmix
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Removes the restriction of specifying group names in users-list file for Gridmix in RoundRobinUserResolver mode.

      Description

      RoundRobinUserResolver.parseUserList() has a bug in obtaining list of groups for each user — in the sense that the list is not cleared before obtaining groups list for the next user. So if the first line has some groups, then from 2nd
      line onwards, the validation of "whether the users(in the next lines) are also having group names in those lines" is useless as the list is already nonempty.

      For example, users-list-file content as shown below also is valid as per parseUserList():
      ------------------
      user1,group1
      user2,
      user3,
      user4,
      ------------------

      1. 2416.patch
        8 kB
        Ravi Gummadi
      2. 2416.v1.patch
        8 kB
        Ravi Gummadi

        Activity

        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk #669 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/669/)

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk #669 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk/669/ )
        Hide
        Hudson added a comment -

        Integrated in Hadoop-Mapreduce-trunk-Commit #652 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/652/)
        MAPREDUCE-2416. Remove the restriction of specifying group names in users-list file for Gridmix in RoundRobinUserResolver mode.

        Show
        Hudson added a comment - Integrated in Hadoop-Mapreduce-trunk-Commit #652 (See https://builds.apache.org/hudson/job/Hadoop-Mapreduce-trunk-Commit/652/ ) MAPREDUCE-2416 . Remove the restriction of specifying group names in users-list file for Gridmix in RoundRobinUserResolver mode.
        Hide
        Ravi Gummadi added a comment -

        I just committed this to trunk.

        Show
        Ravi Gummadi added a comment - I just committed this to trunk.
        Hide
        Ravi Gummadi added a comment -

        Failed contrib tests are not related to this patch at all. I will commit this patch.

        Show
        Ravi Gummadi added a comment - Failed contrib tests are not related to this patch at all. I will commit this patch.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12477630/2416.v1.patch
        against trunk revision 1097345.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.

        +1 release audit. The applied patch does not increase the total number of release audit warnings.

        +1 core tests. The patch passed core unit tests.

        -1 contrib tests. The patch failed contrib unit tests.

        +1 system test framework. The patch passed system test framework compile.

        Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/193//testReport/
        Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/193//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/193//console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12477630/2416.v1.patch against trunk revision 1097345. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. +1 release audit. The applied patch does not increase the total number of release audit warnings. +1 core tests. The patch passed core unit tests. -1 contrib tests. The patch failed contrib unit tests. +1 system test framework. The patch passed system test framework compile. Test results: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/193//testReport/ Findbugs warnings: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/193//artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Console output: https://builds.apache.org/hudson/job/PreCommit-MAPREDUCE-Build/193//console This message is automatically generated.
        Hide
        Amar Kamat added a comment -

        Patch looks good to me. +1.

        Show
        Amar Kamat added a comment - Patch looks good to me. +1.
        Hide
        Ravi Gummadi added a comment -

        Attaching new patch making some minor changes to the comments and documentation.

        Show
        Ravi Gummadi added a comment - Attaching new patch making some minor changes to the comments and documentation.
        Hide
        Ravi Gummadi added a comment -

        Attaching patch that removes the restriction of specifying groups of each user in the users-list-file of RoundRobinUserResolver.

        With this patch, the new format of each line of the users-list-file is

        username[,group]*
        

        [,group]* is just to allow older users-list-files for backward compatibility. Later, may be in the next release, we can fully remove the groups from users-list-file.

        This patch is on top of the patch of MAPREDUCE-2417.

        Show
        Ravi Gummadi added a comment - Attaching patch that removes the restriction of specifying groups of each user in the users-list-file of RoundRobinUserResolver. With this patch, the new format of each line of the users-list-file is username[,group]* [,group] * is just to allow older users-list-files for backward compatibility. Later, may be in the next release, we can fully remove the groups from users-list-file. This patch is on top of the patch of MAPREDUCE-2417 .
        Hide
        Ravi Gummadi added a comment -

        Because the changes for the fix of MAPREDUCE-2417 and the code changes of this JIRA are at the same place in code, the combined patch of MAPREDUCE-2416 and MAPREDUCE-2417 is uploaded to MAPREDUCE-2417.

        Show
        Ravi Gummadi added a comment - Because the changes for the fix of MAPREDUCE-2417 and the code changes of this JIRA are at the same place in code, the combined patch of MAPREDUCE-2416 and MAPREDUCE-2417 is uploaded to MAPREDUCE-2417 .
        Hide
        Ravi Gummadi added a comment -

        Since Gridmix doesn't use/need the groups specified in users-list-file, I am planning to remove the restriction of specifying the groups in this file for RoundRobinUserResolver.

        So the new format of the users-list-file expected is

        user1
        user2
        user3
        ....

        Just to allow backward compatibility, the list of groups can be there after user name after comma in any/all the lines.
        So each line can be of the form

        username[,group]*

        Show
        Ravi Gummadi added a comment - Since Gridmix doesn't use/need the groups specified in users-list-file, I am planning to remove the restriction of specifying the groups in this file for RoundRobinUserResolver. So the new format of the users-list-file expected is user1 user2 user3 .... Just to allow backward compatibility, the list of groups can be there after user name after comma in any/all the lines. So each line can be of the form username [,group] *

          People

          • Assignee:
            Ravi Gummadi
            Reporter:
            Ravi Gummadi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development