Hadoop Common
  1. Hadoop Common
  2. HADOOP-4806

HDFS rename does not work correctly if src contains Java regular expression special characters

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.18.2
    • Fix Version/s: 0.18.3
    • Component/s: None
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      For examples,

      1. create a file /a+b/foo but not close it
      2. rename /a+b/foo to /a+b/bar/foo
        Then, the corresponding INodeFileUnderConstruction will be moved from /a+b/foo to /a+b/bar/foo but the lease path remains unchanged.
      1. 4806_20081208_0.18.patch
        14 kB
        Tsz Wo Nicholas Sze
      2. 4806_20081208.patch
        2 kB
        Tsz Wo Nicholas Sze
      3. 4806_20081209_0.18.patch
        14 kB
        Tsz Wo Nicholas Sze
      4. 4806_20081209.patch
        1 kB
        Tsz Wo Nicholas Sze

        Issue Links

          Activity

          Hide
          Tsz Wo Nicholas Sze added a comment -

          In LeaseManager.changeLease(...), the variable overwrite is not a regular expression pattern but is used as a regular expression pattern in

                final String newpath = oldpath.replaceFirst(overwrite, replaceBy);
          
          Show
          Tsz Wo Nicholas Sze added a comment - In LeaseManager.changeLease(...), the variable overwrite is not a regular expression pattern but is used as a regular expression pattern in final String newpath = oldpath.replaceFirst(overwrite, replaceBy);
          Hide
          Tsz Wo Nicholas Sze added a comment -

          4806_20081208.patch: fixed string replacement bug.

          Show
          Tsz Wo Nicholas Sze added a comment - 4806_20081208.patch: fixed string replacement bug.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          S3FileSystem.renameRecursive(...) also has this problem.

          Show
          Tsz Wo Nicholas Sze added a comment - S3FileSystem.renameRecursive(...) also has this problem.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          4806_20081208_0.18.patch: for 0.18 (added TestRenameWhileOpen)

          Show
          Tsz Wo Nicholas Sze added a comment - 4806_20081208_0.18.patch: for 0.18 (added TestRenameWhileOpen)
          Hide
          Hairong Kuang added a comment -

          Could we Pattern#quote "overwrite"? I just want to be safe...

          Show
          Hairong Kuang added a comment - Could we Pattern#quote "overwrite"? I just want to be safe...
          Hide
          Tsz Wo Nicholas Sze added a comment -

          4806_20081209.patch: using Pattern.quote(..).

          Show
          Tsz Wo Nicholas Sze added a comment - 4806_20081209.patch: using Pattern.quote(..).
          Hide
          Tsz Wo Nicholas Sze added a comment -

          4806_20081209_0.18.patch: for 0.18

          Show
          Tsz Wo Nicholas Sze added a comment - 4806_20081209_0.18.patch: for 0.18
          Hide
          Hairong Kuang added a comment -

          +1

          Show
          Hairong Kuang added a comment - +1
          Hide
          Tsz Wo Nicholas Sze added a comment -
               [exec] +1 overall.  
          
               [exec]     +1 @author.  The patch does not contain any @author tags.
          
               [exec]     +1 tests included.  The patch appears to include 3 new or modified tests.
          
               [exec]     +1 javadoc.  The javadoc tool did not generate any warning messages.
          
               [exec]     +1 javac.  The applied patch does not increase the total number of javac compiler warnings.
          
               [exec]     +1 findbugs.  The patch does not introduce any new Findbugs warnings.
          
               [exec]     +1 Eclipse classpath. The patch retains Eclipse classpath integrity.
          

          Running tests locally.

          Show
          Tsz Wo Nicholas Sze added a comment - [exec] +1 overall. [exec] +1 @author. The patch does not contain any @author tags. [exec] +1 tests included. The patch appears to include 3 new or modified tests. [exec] +1 javadoc. The javadoc tool did not generate any warning messages. [exec] +1 javac. The applied patch does not increase the total number of javac compiler warnings. [exec] +1 findbugs. The patch does not introduce any new Findbugs warnings. [exec] +1 Eclipse classpath. The patch retains Eclipse classpath integrity. Running tests locally.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          Only org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.TestStartAtOffset failed and it is not related to this patch.

          Show
          Tsz Wo Nicholas Sze added a comment - Only org.apache.hadoop.chukwa.datacollection.adaptor.filetailer.TestStartAtOffset failed and it is not related to this patch.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          4806_20081209_0.18.patch: the previous file was wrong. Should be this one.

          Show
          Tsz Wo Nicholas Sze added a comment - 4806_20081209_0.18.patch: the previous file was wrong. Should be this one.
          Hide
          Tsz Wo Nicholas Sze added a comment -

          I just committed this.

          Show
          Tsz Wo Nicholas Sze added a comment - I just committed this.
          Hide
          Hudson added a comment -

          Integrated in Hadoop-trunk #685 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/685/)
          . HDFS rename should not use src path as a regular expression. (szetszwo)

          Show
          Hudson added a comment - Integrated in Hadoop-trunk #685 (See http://hudson.zones.apache.org/hudson/job/Hadoop-trunk/685/ ) . HDFS rename should not use src path as a regular expression. (szetszwo)
          Hide
          Tsz Wo Nicholas Sze added a comment -

          It turned out that the first patch is safer. See HADOOP-6017.

          Show
          Tsz Wo Nicholas Sze added a comment - It turned out that the first patch is safer. See HADOOP-6017 .

            People

            • Assignee:
              Tsz Wo Nicholas Sze
              Reporter:
              Tsz Wo Nicholas Sze
            • Votes:
              0 Vote for this issue
              Watchers:
              0 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development