Hadoop Common
  1. Hadoop Common
  2. HADOOP-4395

Reloading FSImage and FSEditLog may erase user and group information

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.18.2
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      The bug can be reproduced as following:

      bash-3.2$ ./bin/hadoop fs -lsr                 
      -rw-r--r--   3 tsz supergroup       1366 2008-10-10 17:52 /user/tsz/a.txt
      -rw-r--r--   3 tsz supergroup       1366 2008-10-10 17:52 /user/tsz/b.txt
      bash-3.2$ ./bin/hadoop fs -chown sze a.txt
      bash-3.2$ ./bin/hadoop fs -lsr
      -rw-r--r--   3 sze supergroup       1366 2008-10-10 17:52 /user/tsz/a.txt
      -rw-r--r--   3 tsz supergroup       1366 2008-10-10 17:52 /user/tsz/b.txt
      

      Then, restart cluster ...

      bash-3.2$ ./bin/hadoop fs -lsr
      -rw-r--r--   3 sze                  1366 2008-10-10 17:52 /user/tsz/a.txt
      -rw-r--r--   3 tsz supergroup       1366 2008-10-10 17:52 /user/tsz/b.txt
      

      The group information for a.txt is missing.

      1. 4395_20081011.patch
        5 kB
        Tsz Wo Nicholas Sze
      2. 4395_20081011_0.18.patch
        5 kB
        Tsz Wo Nicholas Sze
      3. 4395_20081010.patch
        1 kB
        Tsz Wo Nicholas Sze

        Activity

        Hide
        Tsz Wo Nicholas Sze added a comment -

        HADOOP-3364 changes FSImage and FSEditLog loading. The OP_SET_OWNER command is misinterpreted.

        Show
        Tsz Wo Nicholas Sze added a comment - HADOOP-3364 changes FSImage and FSEditLog loading. The OP_SET_OWNER command is misinterpreted.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        4395_20081010.patch: fix the problem. see whether I could add a unit test.

        Show
        Tsz Wo Nicholas Sze added a comment - 4395_20081010.patch: fix the problem. see whether I could add a unit test.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        Tested 4395_20081010.patch manually. It fixed the problem. Submitting ...

        Show
        Tsz Wo Nicholas Sze added a comment - Tested 4395_20081010.patch manually. It fixed the problem. Submitting ...
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12391924/4395_20081010.patch
        against trunk revision 703609.

        +1 @author. The patch does not contain any @author tags.

        -1 tests included. The patch doesn't appear to include any new or modified tests.
        Please justify why no tests are needed for this patch.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12391924/4395_20081010.patch against trunk revision 703609. +1 @author. The patch does not contain any @author tags. -1 tests included. The patch doesn't appear to include any new or modified tests. Please justify why no tests are needed for this patch. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3445/console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment - - edited

        4395_20081011.patch: updated a test.

        Show
        Tsz Wo Nicholas Sze added a comment - - edited 4395_20081011.patch: updated a test.
        Hide
        Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12391936/4395_20081011.patch
        against trunk revision 703609.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        +1 javadoc. The javadoc tool did not generate any warning messages.

        +1 javac. The applied patch does not increase the total number of javac compiler warnings.

        +1 findbugs. The patch does not introduce any new Findbugs warnings.

        +1 Eclipse classpath. The patch retains Eclipse classpath integrity.

        -1 core tests. The patch failed core unit tests.

        +1 contrib tests. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12391936/4395_20081011.patch against trunk revision 703609. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. +1 javadoc. The javadoc tool did not generate any warning messages. +1 javac. The applied patch does not increase the total number of javac compiler warnings. +1 findbugs. The patch does not introduce any new Findbugs warnings. +1 Eclipse classpath. The patch retains Eclipse classpath integrity. -1 core tests. The patch failed core unit tests. +1 contrib tests. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/3447/console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        The failed core test, TestJobQueueInformation, is not related to this issue. See HADOOP-4378.

        Show
        Tsz Wo Nicholas Sze added a comment - The failed core test, TestJobQueueInformation, is not related to this issue. See HADOOP-4378 .
        Hide
        Chris Douglas added a comment -

        +1 Patch looks good

        Show
        Chris Douglas added a comment - +1 Patch looks good
        Hide
        Tsz Wo Nicholas Sze added a comment -

        4395_20081011_0.18.patch: for 0.18

        Show
        Tsz Wo Nicholas Sze added a comment - 4395_20081011_0.18.patch: for 0.18
        Hide
        Tsz Wo Nicholas Sze added a comment -

        I just committed this.

        Show
        Tsz Wo Nicholas Sze added a comment - I just committed this.
        Hide
        Raghu Angadi added a comment -

        I think a considerable part of the reason why this bug got in is lack of javadoc contract for setOwner() methods. Though these are supposed to be hdfs internal, javadoc should have been there. But this comment is too late for this jira I guess.

        Show
        Raghu Angadi added a comment - I think a considerable part of the reason why this bug got in is lack of javadoc contract for setOwner() methods. Though these are supposed to be hdfs internal, javadoc should have been there. But this comment is too late for this jira I guess.
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #634 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/634/ )

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            1 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development