Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11201

Hadoop Archives should support globs resolving to files

    XMLWordPrintableJSON

    Details

    • Type: Bug
    • Status: Closed
    • Priority: Blocker
    • Resolution: Fixed
    • Affects Version/s: 2.0.5-alpha
    • Fix Version/s: 2.7.0
    • Component/s: tools
    • Labels:
      None
    • Target Version/s:
    • Hadoop Flags:
      Reviewed

      Description

      Consider the following scenario:

      $ hadoop fs -ls /tmp/harsrc/dir2/dir3
      Found 5 items
      -rw-r--r--   1 blah blah          0 2014-10-13 20:59 /tmp/harsrc/dir2/dir3/file31
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file32
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file33
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file34
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file35
      

      Archive 'dir3/file3*':

      $ hadoop archive -Dmapreduce.framework.name=local -archiveName fileStar.har -p /tmp/harsrc 'dir2/dir3/file*' /tmp/hardst_local
      $ hadoop fs -ls -R har:/tmp/hardst_local/fileStar.har
      drwxr-xr-x   - blah blah          0 2014-10-13 22:32 har:///tmp/hardst_local/fileStar.har/dir2
      

      Archiving dir3 (directory) which is equivalent to the above works.

      $ hadoop archive -Dmapreduce.framework.name=local -archiveName dir3.har -p /tmp/harsrc 'dir2/dir3' /tmp/hardst_local
      $ hadoop fs -ls -R har:/tmp/hardst_local/dir3.har
      14/10/14 02:06:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      drwxr-xr-x   - blah blah          0 2014-10-13 22:32 har:///tmp/hardst_local/dir3.har/dir2
      drwxr-xr-x   - blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3
      -rw-r--r--   1 blah blah          0 2014-10-13 20:59 har:///tmp/hardst_local/dir3.har/dir2/dir3/file31
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file32
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file33
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file34
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file35
      

        Attachments

        1. HADOOP-11201.v02.patch
          7 kB
          Gera Shegalov
        2. HADOOP-11201.v01.patch
          6 kB
          Gera Shegalov

          Activity

            People

            • Assignee:
              jira.shegalov Gera Shegalov
              Reporter:
              jira.shegalov Gera Shegalov
            • Votes:
              0 Vote for this issue
              Watchers:
              9 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved: