Uploaded image for project: 'Hadoop Common'
  1. Hadoop Common
  2. HADOOP-11201

Hadoop Archives should support globs resolving to files

    XMLWordPrintableJSON

Details

    • Bug
    • Status: Closed
    • Blocker
    • Resolution: Fixed
    • 2.0.5-alpha
    • 2.7.0
    • tools
    • None
    • Reviewed

    Description

      Consider the following scenario:

      $ hadoop fs -ls /tmp/harsrc/dir2/dir3
      Found 5 items
      -rw-r--r--   1 blah blah          0 2014-10-13 20:59 /tmp/harsrc/dir2/dir3/file31
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file32
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file33
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file34
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 /tmp/harsrc/dir2/dir3/file35
      

      Archive 'dir3/file3*':

      $ hadoop archive -Dmapreduce.framework.name=local -archiveName fileStar.har -p /tmp/harsrc 'dir2/dir3/file*' /tmp/hardst_local
      $ hadoop fs -ls -R har:/tmp/hardst_local/fileStar.har
      drwxr-xr-x   - blah blah          0 2014-10-13 22:32 har:///tmp/hardst_local/fileStar.har/dir2
      

      Archiving dir3 (directory) which is equivalent to the above works.

      $ hadoop archive -Dmapreduce.framework.name=local -archiveName dir3.har -p /tmp/harsrc 'dir2/dir3' /tmp/hardst_local
      $ hadoop fs -ls -R har:/tmp/hardst_local/dir3.har
      14/10/14 02:06:33 WARN util.NativeCodeLoader: Unable to load native-hadoop library for your platform... using builtin-java classes where applicable
      drwxr-xr-x   - blah blah          0 2014-10-13 22:32 har:///tmp/hardst_local/dir3.har/dir2
      drwxr-xr-x   - blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3
      -rw-r--r--   1 blah blah          0 2014-10-13 20:59 har:///tmp/hardst_local/dir3.har/dir2/dir3/file31
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file32
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file33
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file34
      -rw-r--r--   1 blah blah          0 2014-10-14 01:51 har:///tmp/hardst_local/dir3.har/dir2/dir3/file35
      

      Attachments

        1. HADOOP-11201.v01.patch
          6 kB
          Gera Shegalov
        2. HADOOP-11201.v02.patch
          7 kB
          Gera Shegalov

        Activity

          People

            jira.shegalov Gera Shegalov
            jira.shegalov Gera Shegalov
            Votes:
            0 Vote for this issue
            Watchers:
            9 Start watching this issue

            Dates

              Created:
              Updated:
              Resolved: