Hadoop Map/Reduce
  1. Hadoop Map/Reduce
  2. MAPREDUCE-1806

CombineFileInputFormat does not work with paths not on default FS

    Details

    • Type: Bug Bug
    • Status: Closed
    • Priority: Major Major
    • Resolution: Fixed
    • Affects Version/s: 0.22.0, 0.23.1
    • Fix Version/s: 1.2.0, 2.0.3-alpha, 0.23.5
    • Component/s: harchive
    • Labels:
      None
    • Hadoop Flags:
      Reviewed

      Description

      In generating the splits in CombineFileInputFormat, the scheme and authority are stripped out. This creates problems when trying to access the files while generating the splits, as without the har:/, the file won't be accessed through the HarFileSystem.

      1. MAPREDUCE-1806.patch
        1.0 kB
        Gera Shegalov
      2. MAPREDUCE-1806.rev2.patch
        3 kB
        Gera Shegalov
      3. MAPREDUCE-1806.rev3.patch
        3 kB
        Gera Shegalov
      4. MAPREDUCE-1806-branch-1.patch
        3 kB
        Sandy Ryza
      5. MAPREDUCE-1806-branch-1.rev1.patch
        3 kB
        Sandy Ryza
      6. MAPREDUCE-1806.rev4.patch
        4 kB
        Sandy Ryza

        Issue Links

          Activity

          Chris Nauroth made changes -
          Link This issue is related to MAPREDUCE-5161 [ MAPREDUCE-5161 ]
          Thomas Graves made changes -
          Status Resolved [ 5 ] Closed [ 6 ]
          Thomas Graves made changes -
          Fix Version/s 0.23.5 [ 12323312 ]
          Alejandro Abdelnur made changes -
          Status Patch Available [ 10002 ] Resolved [ 5 ]
          Hadoop Flags Reviewed [ 10343 ]
          Fix Version/s 1.2.0 [ 12321661 ]
          Fix Version/s 2.0.3-alpha [ 12323275 ]
          Resolution Fixed [ 1 ]
          Alejandro Abdelnur made changes -
          Assignee Gera Shegalov [ jira.shegalov ]
          Sandy Ryza made changes -
          Attachment MAPREDUCE-1806.rev4.patch [ 12551273 ]
          Sandy Ryza made changes -
          Attachment MAPREDUCE-1806.rev4.patch [ 12551257 ]
          Sandy Ryza made changes -
          Attachment MAPREDUCE-1806-branch-1.rev1.patch [ 12551272 ]
          Sandy Ryza made changes -
          Attachment MAPREDUCE-1806.rev4.patch [ 12551257 ]
          Sandy Ryza made changes -
          Attachment MAPREDUCE-1806-branch-1.patch [ 12550366 ]
          Gera Shegalov made changes -
          Status Open [ 1 ] Patch Available [ 10002 ]
          Gera Shegalov made changes -
          Attachment MAPREDUCE-1806.rev3.patch [ 12546603 ]
          Gera Shegalov made changes -
          Attachment MAPREDUCE-1806.rev2.patch [ 12546264 ]
          Gera Shegalov made changes -
          Attachment MAPREDUCE-1806.patch [ 12545834 ]
          Todd Lipcon made changes -
          Summary CombineFileInputFormat does not work with HAR paths CombineFileInputFormat does not work with paths not on default FS
          Vinod Kumar Vavilapalli made changes -
          Link This issue duplicates MAPREDUCE-2704 [ MAPREDUCE-2704 ]
          Carl Steinbach made changes -
          Link This issue relates to HIVE-3025 [ HIVE-3025 ]
          Carl Steinbach made changes -
          Field Original Value New Value
          Affects Version/s 0.23.1 [ 12318883 ]
          Paul Yang created issue -

            People

            • Assignee:
              Gera Shegalov
              Reporter:
              Paul Yang
            • Votes:
              0 Vote for this issue
              Watchers:
              15 Start watching this issue

              Dates

              • Created:
                Updated:
                Resolved:

                Development