Hadoop Common
  1. Hadoop Common
  2. HADOOP-1961

-get, -copyToLocal fail when single filename is passed

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Blocker Blocker
    • Resolution: Fixed
    • Affects Version/s: 0.14.1
    • Fix Version/s: 0.14.2, 0.15.0
    • Component/s: None
    • Labels:
      None

      Description

      In 0.14.1 and in trunk, when I try

      % hadoop dfs -get /user/knoguchi/aaa aaa

      get: Failed to rename tmp file to local destination "aaa". Remote source file "/user/knoguchi/aaa" is saved to "/tmp/_copyToLocal_aaa30478".

      This works.

      % hadoop dfs -get /user/knoguchi/aaa ./aaa

      or

      % hadoop dfs -get /user/knoguchi/aaa /home/knoguchi/aaa

      My guess. With change in HADOOP-1292, it now creates a tmp file when -copyToLocal.
      When destination path is passed without any directory, tmp file is created under '/tmp'. Otherwise, it uses the same directory as the destination path.

      In Java API for File.renameTo,
      http://java.sun.com/javase/6/docs/api/java/io/File.html#renameTo(java.io.File)
      it says
      " The rename operation might not be able to move a file from one filesystem to another",

      so renameTo call from /tmp/_tmpfile to /home/knoguchi can fail.

      1. HADOOP-1961-branch14.patch
        5 kB
        Raghu Angadi
      2. HADOOP-1961-branch14.patch
        8 kB
        Raghu Angadi
      3. HADOOP-1961.patch
        2 kB
        Raghu Angadi
      4. HADOOP-1961.patch
        5 kB
        Raghu Angadi
      5. HADOOP-1961.patch
        5 kB
        Raghu Angadi
      6. HADOOP-1961.patch
        8 kB
        Raghu Angadi

        Activity

        Koji Noguchi created issue -
        dhruba borthakur made changes -
        Field Original Value New Value
        Assignee Raghu Angadi [ rangadi ]
        Koji Noguchi made changes -
        Fix Version/s 0.14.2 [ 12312762 ]
        Raghu Angadi made changes -
        Attachment HADOOP-1961.patch [ 12366717 ]
        Raghu Angadi made changes -
        Attachment HADOOP-1961.patch [ 12366789 ]
        Raghu Angadi made changes -
        Attachment HADOOP-1961-branch14.patch [ 12367034 ]
        Raghu Angadi made changes -
        Attachment HADOOP-1961.patch [ 12367039 ]
        Raghu Angadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        Raghu Angadi made changes -
        Status Patch Available [ 10002 ] Open [ 1 ]
        Raghu Angadi made changes -
        Attachment HADOOP-1961-branch14.patch [ 12367088 ]
        Raghu Angadi made changes -
        Attachment HADOOP-1961.patch [ 12367089 ]
        Raghu Angadi made changes -
        Status Open [ 1 ] Patch Available [ 10002 ]
        dhruba borthakur made changes -
        Resolution Fixed [ 1 ]
        Status Patch Available [ 10002 ] Resolved [ 5 ]
        Fix Version/s 0.15.0 [ 12312565 ]
        Doug Cutting made changes -
        Status Resolved [ 5 ] Closed [ 6 ]
        Owen O'Malley made changes -
        Component/s dfs [ 12310710 ]

          People

          • Assignee:
            Raghu Angadi
            Reporter:
            Koji Noguchi
          • Votes:
            0 Vote for this issue
            Watchers:
            0 Start watching this issue

            Dates

            • Created:
              Updated:
              Resolved:

              Development