Details

    • Type: Improvement
    • Status: Closed
    • Priority: Major
    • Resolution: Fixed
    • Affects Version/s: 0.20.2, 0.20.205.0
    • Fix Version/s: 0.20.205.0
    • Component/s: None
    • Labels:
      None

      Description

      HADOOP-7110 implemented chmod in the NativeIO library so we can have good performance (ie not fork) and still not be prone to races. This should fix build failures (and probably task failures too).

      1. HADOOP-7432.patch
        5 kB
        Sherry Chen
      2. HADOOP-7432_1.patch
        5 kB
        Sherry Chen
      3. HADOOP-7432_2.patch
        5 kB
        Sherry Chen

        Activity

        Hide
        mattf Matt Foley added a comment -

        Closed upon release of 0.20.205.0

        Show
        mattf Matt Foley added a comment - Closed upon release of 0.20.205.0
        Hide
        mattf Matt Foley added a comment -

        Todd said in email:

        Feel free to +1 and commit.

        Committed to 0.20-security. Thanks Sherry! And thanks Todd for the reviews.

        Show
        mattf Matt Foley added a comment - Todd said in email: Feel free to +1 and commit. Committed to 0.20-security. Thanks Sherry! And thanks Todd for the reviews.
        Hide
        sherri_chen Sherry Chen added a comment -

        Results of running test-patch locally on this new patch:

        [exec]
        [exec] BUILD SUCCESSFUL
        [exec] Total time: 6 minutes 6 seconds
        [exec]
        [exec]
        [exec]
        [exec]
        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
        [exec]
        [exec]
        [exec]
        [exec]
        [exec] ======================================================================
        [exec] ======================================================================
        [exec] Finished build.
        [exec] ======================================================================
        [exec] ======================================================================
        [exec]

        Show
        sherri_chen Sherry Chen added a comment - Results of running test-patch locally on this new patch: [exec] [exec] BUILD SUCCESSFUL [exec] Total time: 6 minutes 6 seconds [exec] [exec] [exec] [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ====================================================================== [exec]
        Hide
        mattf Matt Foley added a comment -

        @Sherry, can you please paste into a comment the results of running test-patch locally on this new patch?

        @Todd, does this version pass code review? Thanks.

        Show
        mattf Matt Foley added a comment - @Sherry, can you please paste into a comment the results of running test-patch locally on this new patch? @Todd, does this version pass code review? Thanks.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12486256/HADOOP-7432_2.patch
        against trunk revision 1145839.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/722//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12486256/HADOOP-7432_2.patch against trunk revision 1145839. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/722//console This message is automatically generated.
        Hide
        sherri_chen Sherry Chen added a comment -

        Tabs in RawLocalFileSystem is removed.

        Show
        sherri_chen Sherry Chen added a comment - Tabs in RawLocalFileSystem is removed.
        Hide
        tlipcon Todd Lipcon added a comment -

        this patch introduces some tabs in RawLocalFileSystem

        otherwise seems good.

        Show
        tlipcon Todd Lipcon added a comment - this patch introduces some tabs in RawLocalFileSystem otherwise seems good.
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12484801/HADOOP-7432_1.patch
        against trunk revision 1141415.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/688//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12484801/HADOOP-7432_1.patch against trunk revision 1141415. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/688//console This message is automatically generated.
        Hide
        sherri_chen Sherry Chen added a comment -

        Agree.
        Line 130 of RawLocalFileSystem.java reverted back to "%04o".

        Show
        sherri_chen Sherry Chen added a comment - Agree. Line 130 of RawLocalFileSystem.java reverted back to "%04o".
        Hide
        revans2 Robert Joseph Evans added a comment -

        It looks like the change was made as part of HADOOP-3953. Implement sticky bit for directories in HDFS. (Jakob Homan via szetszwo)

        There were a lot of other changes too associated with that patch, could you please revert it back to "%04o". It should not make any difference in the actual operation of the chmod command, but it will be more consistent with what is actually supported by 0.20-security.

        Show
        revans2 Robert Joseph Evans added a comment - It looks like the change was made as part of HADOOP-3953 . Implement sticky bit for directories in HDFS. (Jakob Homan via szetszwo) There were a lot of other changes too associated with that patch, could you please revert it back to "%04o". It should not make any difference in the actual operation of the chmod command, but it will be more consistent with what is actually supported by 0.20-security.
        Hide
        sherri_chen Sherry Chen added a comment -

        Copied from HADOOP:7110.
        Seems trunk has "%05o" but 0.20-security has "%04o".

        Show
        sherri_chen Sherry Chen added a comment - Copied from HADOOP:7110. Seems trunk has "%05o" but 0.20-security has "%04o".
        Hide
        revans2 Robert Joseph Evans added a comment -

        Why did line 130 of RawLocalFileSystem.java change from "%04o" to "%05o"?

        Show
        revans2 Robert Joseph Evans added a comment - Why did line 130 of RawLocalFileSystem.java change from "%04o" to "%05o"?
        Hide
        hadoopqa Hadoop QA added a comment -

        -1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12484697/HADOOP-7432.patch
        against trunk revision 1140442.

        +1 @author. The patch does not contain any @author tags.

        +1 tests included. The patch appears to include 3 new or modified tests.

        -1 patch. The patch command could not apply the patch.

        Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/685//console

        This message is automatically generated.

        Show
        hadoopqa Hadoop QA added a comment - -1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12484697/HADOOP-7432.patch against trunk revision 1140442. +1 @author. The patch does not contain any @author tags. +1 tests included. The patch appears to include 3 new or modified tests. -1 patch. The patch command could not apply the patch. Console output: https://builds.apache.org/job/PreCommit-HADOOP-Build/685//console This message is automatically generated.
        Hide
        sherri_chen Sherry Chen added a comment -

        Manual test result:

        [exec]
        [exec] +1 overall.
        [exec]
        [exec] +1 @author. The patch does not contain any @author tags.
        [exec]
        [exec] +1 tests included. The patch appears to include 3 new or modified tests.
        [exec]
        [exec] +1 javadoc. The javadoc tool did not generate any warning messages.
        [exec]
        [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings.
        [exec]
        [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings.
        [exec]
        [exec]
        [exec]
        [exec]
        [exec] ======================================================================
        [exec] ======================================================================
        [exec] Finished build.
        [exec] ======================================================================
        [exec] ======================================================================

        Show
        sherri_chen Sherry Chen added a comment - Manual test result: [exec] [exec] +1 overall. [exec] [exec] +1 @author. The patch does not contain any @author tags. [exec] [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] [exec] +1 findbugs. The patch does not introduce any new Findbugs (version 1.3.9) warnings. [exec] [exec] [exec] [exec] [exec] ====================================================================== [exec] ====================================================================== [exec] Finished build. [exec] ====================================================================== [exec] ======================================================================
        Hide
        sherri_chen Sherry Chen added a comment -

        Back-ported HADOOP-7110 to 0.20-security.

        Show
        sherri_chen Sherry Chen added a comment - Back-ported HADOOP-7110 to 0.20-security.

          People

          • Assignee:
            sherri_chen Sherry Chen
            Reporter:
            sherri_chen Sherry Chen
          • Votes:
            0 Vote for this issue
            Watchers:
            3 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development