Bigtop
  1. Bigtop
  2. BIGTOP-997

TestCLI fails test id 5: comparator fails

    Details

    • Type: Bug Bug
    • Status: Open
    • Priority: Major Major
    • Resolution: Unresolved
    • Affects Version/s: 0.5.0
    • Fix Version/s: 0.8.0
    • Component/s: Tests

      Description

      I am running hadoop smokes against a cluster and at least one test case is failing in TestCLI:

      13/06/05 19:22:02 INFO cli.CLITestHelper:                     Test ID: [5]
      ...
      13/06/05 19:22:02 INFO cli.CLITestHelper:          Comparision result:   [fail]
      13/06/05 19:22:02 INFO cli.CLITestHelper:             Expected output:   [^drwxr-xr-x( )*-( )*[a-z]*( )*hadoo
      p( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*dir1]
      13/06/05 19:22:02 INFO cli.CLITestHelper:               Actual output:   [Found 1 items
      drwxr-xr-x   - cos supergroup          0 2013-06-05 19:20 dir1
      

        Activity

        Hide
        Plamen Jeliazkov added a comment - - edited

        This appears to have already been fixed.

        From master branch – the test code:

        <test> <!-- TESTED test-id=5 -->
              <description>ls: directory using relative path</description>
              <test-commands>
                <command>-fs NAMENODE -mkdir -p dir1</command>
                <command>-fs NAMENODE -ls </command>
              </test-commands>
              <cleanup-commands>
                <command>-fs NAMENODE -rm -r /user/USER_NAME/dir1</command>
              </cleanup-commands>
              <comparators>
                <comparator>
                  <type>TokenComparator</type>
                  <expected-output>Found 1 items</expected-output>
                </comparator>
                <comparator>
                  <type>RegexpComparator</type>
                  <expected-output>^drwxr-xr-x( )*-( )*[a-z]*( )*supergroup( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*dir1</expected-output>
                </comparator>
              </comparators>
            </test>
        

        The comparator has already been changed to check for the supergroup group, not the hadoop group. This is safe to close I think.

        Show
        Plamen Jeliazkov added a comment - - edited This appears to have already been fixed. From master branch – the test code: <test> <!-- TESTED test-id=5 --> <description>ls: directory using relative path</description> <test-commands> <command>-fs NAMENODE -mkdir -p dir1</command> <command>-fs NAMENODE -ls </command> </test-commands> <cleanup-commands> <command>-fs NAMENODE -rm -r /user/USER_NAME/dir1</command> </cleanup-commands> <comparators> <comparator> <type>TokenComparator</type> <expected-output>Found 1 items</expected-output> </comparator> <comparator> <type>RegexpComparator</type> <expected-output>^drwxr-xr-x( )*-( )*[a-z]*( )*supergroup( )*0( )*[0-9]{4,}-[0-9]{2,}-[0-9]{2,} [0-9]{2,}:[0-9]{2,}( )*dir1</expected-output> </comparator> </comparators> </test> The comparator has already been changed to check for the supergroup group, not the hadoop group. This is safe to close I think.
        Hide
        Konstantin Boudnik added a comment -

        This is weird bug: I keep seeing this problem although the XML file is correct and the files are created with "superuser" group as expected. For some crazy reason this hadoop expectation is popping from nowhere.

        Show
        Konstantin Boudnik added a comment - This is weird bug: I keep seeing this problem although the XML file is correct and the files are created with "superuser" group as expected. For some crazy reason this hadoop expectation is popping from nowhere.
        Hide
        Konstantin Boudnik added a comment -

        Ok, the problem with the bug is this:

        • as a part of BIGTOP-895 everything that say supergroup is now replaced with whatever coming from DFSConfigKeys.DFS_PERMISSIONS_SUPERUSERGROUP_KEY
        • by default it is set to be group hadoop
        • when TestCLI creates files and directories under /tmp they are inherit group settings from the parent directory (which is hadoop normally)
        • when directory or file creation happens with a relative path (e.g. fs -mkdir -p dir1 the new directory is getting created under user's home e.g. /user/jenkins which is likely to have supergroup group credentials.

        This eventually leads to the situation where we see the bug if the test is ran under user belonging to group supergroup. One way to fix it is to make sure that user's group is hadoop. Another way is to have smarter expandCommand in the test.

        Show
        Konstantin Boudnik added a comment - Ok, the problem with the bug is this: as a part of BIGTOP-895 everything that say supergroup is now replaced with whatever coming from DFSConfigKeys.DFS_PERMISSIONS_SUPERUSERGROUP_KEY by default it is set to be group hadoop when TestCLI creates files and directories under /tmp they are inherit group settings from the parent directory (which is hadoop normally) when directory or file creation happens with a relative path (e.g. fs -mkdir -p dir1 the new directory is getting created under user's home e.g. /user/jenkins which is likely to have supergroup group credentials. This eventually leads to the situation where we see the bug if the test is ran under user belonging to group supergroup . One way to fix it is to make sure that user's group is hadoop . Another way is to have smarter expandCommand in the test.
        Hide
        Konstantin Boudnik added a comment -

        hmm... I also see test id=4 is failing. And that one is working with /tmp directory. Is something totally crazy with my cluster?

        Show
        Konstantin Boudnik added a comment - hmm... I also see test id=4 is failing. And that one is working with /tmp directory. Is something totally crazy with my cluster?

          People

          • Assignee:
            Konstantin Boudnik
            Reporter:
            Konstantin Boudnik
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:

              Development