Hadoop Common
  1. Hadoop Common
  2. HADOOP-3099

Need new options in distcp for preserving ower, group and permission

    Details

    • Type: Improvement Improvement
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: None
    • Fix Version/s: 0.17.0
    • Component/s: util
    • Labels:
      None
    • Hadoop Flags:
      Reviewed
    • Release Note:
      Hide
      Added a new option -p to distcp for preserving file/directory status.
      -p[rbugp] Preserve status
                             r: replication number
                             b: block size
                             u: user
                             g: group
                             p: permission
                             -p alone is equivalent to -prbugp
      Show
      Added a new option -p to distcp for preserving file/directory status. -p[rbugp] Preserve status                        r: replication number                        b: block size                        u: user                        g: group                        p: permission                        -p alone is equivalent to -prbugp

      Description

      Currently, distcp -p preserves replication# and block size. Since permissions are introduced, distcp should provide options for preserving ower, group and permission.

      1. 3099_20080402b.patch
        29 kB
        Tsz Wo Nicholas Sze
      2. 3099_20080402b_0.16.patch
        31 kB
        Tsz Wo Nicholas Sze

        Activity

        Hide
        Tsz Wo Nicholas Sze added a comment -

        3099_20080402b_0.16.patch: for 0.16

        Show
        Tsz Wo Nicholas Sze added a comment - 3099_20080402b_0.16.patch: for 0.16
        Hide
        Hudson added a comment -
        Show
        Hudson added a comment - Integrated in Hadoop-trunk #451 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/451/ )
        Hide
        Chris Douglas added a comment -

        I just committed this. Thanks, Nicholas!

        Show
        Chris Douglas added a comment - I just committed this. Thanks, Nicholas!
        Hide
        Chris Douglas added a comment -

        +1 Looks good.

        Show
        Chris Douglas added a comment - +1 Looks good.
        Hide
        Hadoop QA added a comment -

        +1 overall. Here are the results of testing the latest attachment
        http://issues.apache.org/jira/secure/attachment/12379223/3099_20080402b.patch
        against trunk revision 643282.

        @author +1. The patch does not contain any @author tags.

        tests included +1. The patch appears to include 6 new or modified tests.

        javadoc +1. The javadoc tool did not generate any warning messages.

        javac +1. The applied patch does not generate any new javac compiler warnings.

        release audit +1. The applied patch does not generate any new release audit warnings.

        findbugs +1. The patch does not introduce any new Findbugs warnings.

        core tests +1. The patch passed core unit tests.

        contrib tests +1. The patch passed contrib unit tests.

        Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/testReport/
        Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html
        Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/artifact/trunk/build/test/checkstyle-errors.html
        Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/console

        This message is automatically generated.

        Show
        Hadoop QA added a comment - +1 overall. Here are the results of testing the latest attachment http://issues.apache.org/jira/secure/attachment/12379223/3099_20080402b.patch against trunk revision 643282. @author +1. The patch does not contain any @author tags. tests included +1. The patch appears to include 6 new or modified tests. javadoc +1. The javadoc tool did not generate any warning messages. javac +1. The applied patch does not generate any new javac compiler warnings. release audit +1. The applied patch does not generate any new release audit warnings. findbugs +1. The patch does not introduce any new Findbugs warnings. core tests +1. The patch passed core unit tests. contrib tests +1. The patch passed contrib unit tests. Test results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/testReport/ Findbugs warnings: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/artifact/trunk/build/test/findbugs/newPatchFindbugsWarnings.html Checkstyle results: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/artifact/trunk/build/test/checkstyle-errors.html Console output: http://hudson.zones.apache.org/hudson/job/Hadoop-Patch/2138/console This message is automatically generated.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        3099_20080402b.patch:

        -p[rbugp]              Preserve status
                               r: replication number
                               b: block size
                               u: user
                               g: group
                               p: permission
                               -p alone is equivalent to -prbugp
        
        Show
        Tsz Wo Nicholas Sze added a comment - 3099_20080402b.patch: -p[rbugp] Preserve status r: replication number b: block size u: user g: group p: permission -p alone is equivalent to -prbugp
        Hide
        Robert Chansler added a comment -

        0.17? Yes!

        Show
        Robert Chansler added a comment - 0.17? Yes!
        Hide
        Nigel Daley added a comment -

        Not going to make 0.16.2. Moving this 0.17. Could backport to 0.16.3 if there is one.

        Show
        Nigel Daley added a comment - Not going to make 0.16.2. Moving this 0.17. Could backport to 0.16.3 if there is one.
        Hide
        Tsz Wo Nicholas Sze added a comment -

        messed up the issues

        Show
        Tsz Wo Nicholas Sze added a comment - messed up the issues
        Hide
        Doug Cutting added a comment -

        > -p should include ugp by default.

        +1 This is a change from current behaviour, but I'd argue it's a bugfix rather than an incompatibility and I'd be very surprised if it creates any problems.

        Show
        Doug Cutting added a comment - > -p should include ugp by default. +1 This is a change from current behaviour, but I'd argue it's a bugfix rather than an incompatibility and I'd be very surprised if it creates any problems.
        Hide
        Allen Wittenauer added a comment -

        -p should include ugp by default. Pretty much EVERY standard UNIX command uses -p == permissions (tar, pax, cp, scp, ... ). This is going to be expected behavior. Anything less is going to be extremely confusing and cause endless issues.

        Show
        Allen Wittenauer added a comment - -p should include ugp by default. Pretty much EVERY standard UNIX command uses -p == permissions (tar, pax, cp, scp, ... ). This is going to be expected behavior. Anything less is going to be extremely confusing and cause endless issues.
        Hide
        Sameer Paranjpye added a comment -

        +1

        Show
        Sameer Paranjpye added a comment - +1
        Hide
        Tsz Wo Nicholas Sze added a comment - - edited

        I suggest changing -p to -p[rbugp]:
        r: replication#
        b: block size
        u: user
        g: group
        p: permission

        -p alone means -prb for backward compatibility.

        Show
        Tsz Wo Nicholas Sze added a comment - - edited I suggest changing -p to -p [rbugp] : r: replication# b: block size u: user g: group p: permission -p alone means -prb for backward compatibility.

          People

          • Assignee:
            Tsz Wo Nicholas Sze
            Reporter:
            Tsz Wo Nicholas Sze
          • Votes:
            0 Vote for this issue
            Watchers:
            2 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development